Walk-through: Unmixing

For the YouTube livestream schedule, see here

For screen-shot slides, click here

---

Background

Rationale

This walk-through is an extension of the Week 14 lesson, documenting how the UnmixingInternals() function was cobbled together from start to finish. This walk-through is optional, and has been separated out from the main Unmixing lesson in a desire to avoid burning out those who would rather not know the tiny details of how to reformat the flowFrame internals to convert from a raw to an unmixed .fcs file.

However, if deep-diving how .fcs file internals are formatted in R is your thing, then this walk-through is definitely for you, so feel free to continue reading :D

Journey Thus Far

At this point in the Week 14 lesson, our OldFashionedUnmixing() function is fairly far along, taking as inputs a file.path to our raw full-stained .fcs file, and also a signature matrix, and currently returning as an output a data.frame containing the newly unmixed fluorophore columns.

#' This function unmixes our raw full-stained .fcs files using the signature matrix we provide.
#' 
#' @param x A file.path to a raw full-stained .fcs file we want to unmix. 
#' @param retainThese Default "FSC|SSC|Time", used to separate out columns not used for unmixing,
#' but that should be retained for the final unmixed .fcs files. 
#' @param detectorExclude Default is "-H|-W", intended to remove additional detector columns other
#' than -A, adjust as needed for your own instruments configuration
#' @param SignatureData A data.frame containing a Fluorophore, Antigen and Detector columns. 
#' @param returnType Default "fcs", for residual plots use "residual"
#' 
#' @importFrom flowCore read.FCS exprs
#' @importFrom dplyr select matches where
#' @importFrom utils read.csv
#' 
OldFashionedUnmix <- function(x, retainThese="FSC|SSC|Time",
 detectorExclude="-H|-W", SignatureData, returnType="fcs"){

# Retrieve the Raw Data

    TheRawFCS <- flowCore::read.FCS(filename=x, transformation=FALSE, truncate_max_range = FALSE)
    TheRawMatrix <- flowCore::exprs(TheRawFCS)
    TheRawDataFrame <- data.frame(TheRawMatrix, check.names=FALSE)

    # Identify the Detector Columns

    StashedColumns <- TheRawDataFrame |> dplyr::select(dplyr::matches(retainThese)) 
    WorkingColumns <- TheRawDataFrame |> dplyr::select(!dplyr::matches(retainThese)) 
    WorkingColumns <- WorkingColumns |> dplyr::select(!dplyr::matches(detectorExclude)) 

    # Retrieve the Signature Matrix 

    if(is.data.frame(SignatureData)){
        Signatures <- SignatureData
    } else { 
        Signatures <-read.csv(SignatureData, check.names=FALSE)
    }

    # Separate Metadata from Detector Columns

    Metadata <- Signatures |> dplyr::select(!dplyr::where(is.numeric))
    Numerics <- Signatures |> dplyr::select(dplyr::where(is.numeric))

    # If not already normalized, scale the signature values

    if (any(Numerics > 1)) {
        message("Signature values greater than 1 detected, normalizing")
        n <- Numerics
        # n[n < 0] <- 0
        A <- do.call(pmax, n)
        Normalized <- n/A
        Numerics <- Normalized
    }

    # Verify your signature matrix and raw full-stained have the same number of columns

    if (!all(colnames(Numerics) == colnames(WorkingColumns))){
        stop("colnames of SignatureData due not match the internal colnames of exprs")
    }

    # Transpose both signature and full-stained matrices, and unmix with OLS

    DetectorNameBackups <- colnames(Numerics)
    TransposedSignatureValues <- t(Numerics)
    TransposedSampleValues <- t(WorkingColumns)
    LeastSquares <- lsfit(x = TransposedSignatureValues,
     y = TransposedSampleValues, intercept = FALSE)
    TransposedLeastSquares <- t(LeastSquares$coefficients)

    # Optional fork to return the residual plot instead
    if (returnType == "residuals"){
    Plot <- Hell3(LeastSquaresList = LeastSquares)
    return(Plot)
    }

    # Update the column names
    FluorophoreNames <- Metadata |> pull(Fluorophore)
    TheDetectorColNames <- colnames(WorkingColumns)
    AppendThisLetter <- sub("^[^-]*", "", TheDetectorColNames) |> unique()
    FluorophoreNames <- paste0(FluorophoreNames, AppendThisLetter)
    colnames(TransposedLeastSquares) <- FluorophoreNames

    # Bind the stashed Time, SSC and FSC columns to the new fluorophore columns
    UnmixedData <- bind_cols(StashedColumns, TransposedLeastSquares)

    return(UnmixedData)
}

Now that we have the unmixed fluorophore data, we need to integrate this data.frame object back into the exprs() matrix slot. However, as we encountered in the Week 10 bonus material when adding metadata columns pre-concatenation, addition or removal of a column in the exprs starts a cascade of changes. Consequently, for both the parameter() and keyword() slots we will need to remove any column name that is no longer present (in this case, the detector columns), and then follow up by adding the corresponding equivalent columns for each of the new fluorophore columns that are now present.

As we might anticipate, this means a lot of moving pieces, so this “Unmixing” bonus-walkthrough is going to be failry similar to the the “Concatenate” bonus-walkthrough from Week 10 in scope.

Set Up

To get started, we need to recreate the local environment objects/variables that we had present in the original walk-through (index.qmd) before we relocated the rest of the UnmixingInternals() code to its own separate bonus walk-through (unmixing.qmd). Since its not easy to share things between quarto documents when rendering as a webpage, we will copy over the code-blocks we previously assembled, and re-run them so that we have all the previously present variables within our working environment so that we can continue assembling the UnmixingInternal() function.

As always, we can start by re-attaching the required R packages to our local environment via the library() call.

library(flowWorkspace)

As part of improvements to flowWorkspace, some behavior of
GatingSet objects has changed. For details, please read the section
titled "The cytoframe and cytoset classes" in the package vignette:

  vignette("flowWorkspace-Introduction", "flowWorkspace")

library(Luciernaga)
library(dplyr)

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

library(purrr)

Next up, lets designate the file.path() to both our storage and output folders.

#StorageLocation <- file.path("course", "14_Unmixing", "data")
StorageLocation <- file.path("data")

#OutputLocation <- file.path("course", "14_Unmixing", "outputs")
OutputLocation <- file.path("outputs")

We can then identify both the .csv and .fcs files that we have for this week’s dataset using list.files().

csv_files <- list.files(StorageLocation, pattern=".csv", full.names=TRUE)

fcs_files <- list.files(StorageLocation, pattern=".fcs", full.names=TRUE)

Once this is done, we can load the individual .csv files via read.csv() to have access to our signature matrix datasets. We will also remove a few of the additional metadata columns that are no longer being used.

BeadSignatures <- read.csv(csv_files[1], check.names=FALSE)
BeadSignatures <- BeadSignatures |>
     select(-c(name, Type, Detector, Negative))
colnames(BeadSignatures)

 [1] "Fluorophore" "Antigen"     "UV1-A"       "UV2-A"       "UV3-A"      
 [6] "UV4-A"       "UV5-A"       "UV6-A"       "UV7-A"       "UV8-A"      
[11] "UV9-A"       "UV10-A"      "UV11-A"      "UV12-A"      "UV13-A"     
[16] "UV14-A"      "UV15-A"      "UV16-A"      "V1-A"        "V2-A"       
[21] "V3-A"        "V4-A"        "V5-A"        "V6-A"        "V7-A"       
[26] "V8-A"        "V9-A"        "V10-A"       "V11-A"       "V12-A"      
[31] "V13-A"       "V14-A"       "V15-A"       "V16-A"       "B1-A"       
[36] "B2-A"        "B3-A"        "B4-A"        "B5-A"        "B6-A"       
[41] "B7-A"        "B8-A"        "B9-A"        "B10-A"       "B11-A"      
[46] "B12-A"       "B13-A"       "B14-A"       "YG1-A"       "YG2-A"      
[51] "YG3-A"       "YG4-A"       "YG5-A"       "YG6-A"       "YG7-A"      
[56] "YG8-A"       "YG9-A"       "YG10-A"      "R1-A"        "R2-A"       
[61] "R3-A"        "R4-A"        "R5-A"        "R6-A"        "R7-A"       
[66] "R8-A"       

BeadSignatures |> pull(Fluorophore)

 [1] "BUV395"          "BUV496"          "BUV563"          "BUV615"         
 [5] "BUV661"          "BUV737"          "BUV805"          "BV421"          
 [9] "Pacific Blue"    "BV480"           "BV510"           "BV605"          
[13] "BV650"           "BV711"           "BV750"           "BV786"          
[17] "FITC"            "Spark Blue 550"  "PerCP-Cy5.5"     "PE"             
[21] "PE-Dazzle 594"   "PE-Cy5"          "PE-Vio 770"      "APC"            
[25] "Alexa Fluor 647" "APC-R700"        "APC-Fire 750"    "APC-Fire 810"   

CellSignatures <- read.csv(csv_files[2], check.names=FALSE)
CellSignatures <- CellSignatures |>
     select(-c(name, Type, Detector, Negative))
CellSignatures <- CellSignatures |>
     dplyr::filter(!stringr::str_detect(Fluorophore, "_Unstained"))
colnames(CellSignatures)

 [1] "Fluorophore" "Antigen"     "UV1-A"       "UV2-A"       "UV3-A"      
 [6] "UV4-A"       "UV5-A"       "UV6-A"       "UV7-A"       "UV8-A"      
[11] "UV9-A"       "UV10-A"      "UV11-A"      "UV12-A"      "UV13-A"     
[16] "UV14-A"      "UV15-A"      "UV16-A"      "V1-A"        "V2-A"       
[21] "V3-A"        "V4-A"        "V5-A"        "V6-A"        "V7-A"       
[26] "V8-A"        "V9-A"        "V10-A"       "V11-A"       "V12-A"      
[31] "V13-A"       "V14-A"       "V15-A"       "V16-A"       "B1-A"       
[36] "B2-A"        "B3-A"        "B4-A"        "B5-A"        "B6-A"       
[41] "B7-A"        "B8-A"        "B9-A"        "B10-A"       "B11-A"      
[46] "B12-A"       "B13-A"       "B14-A"       "YG1-A"       "YG2-A"      
[51] "YG3-A"       "YG4-A"       "YG5-A"       "YG6-A"       "YG7-A"      
[56] "YG8-A"       "YG9-A"       "YG10-A"      "R1-A"        "R2-A"       
[61] "R3-A"        "R4-A"        "R5-A"        "R6-A"        "R7-A"       
[66] "R8-A"       

CellSignatures |> pull(Fluorophore)

 [1] "BUV395"          "BUV496"          "BUV563"          "BUV615"         
 [5] "BUV661"          "BUV737"          "BUV805"          "BV421"          
 [9] "Pacific Blue"    "BV480"           "BV510"           "BV605"          
[13] "BV650"           "BV711"           "BV750"           "BV786"          
[17] "FITC"            "Spark Blue 550"  "PerCP-Cy5.5"     "PE"             
[21] "PE-Dazzle 594"   "PE-Cy5"          "PE-Vio 770"      "APC"            
[25] "Alexa Fluor 647" "APC-R700"        "Zombie NIR"      "APC-Fire 750"   
[29] "APC-Fire 810"   

Looking at the fluorophores present in each matrix, we can see we still need to add the signatures for both our “Zombie NIR” and “Unstained”, so that we do not have issues for signal left unaccounted for inflating the residuals or being misatributed to the other fluorophores. We can combine filter() and str_detect() to isolate them out, allowing us to bing them out and bind the individual rows

AlternateSignatures <- read.csv(csv_files[3], check.names=FALSE)
AlternateSignatures <- AlternateSignatures |>
     select(-c(name, Type, Detector, Negative))
Unstained <- AlternateSignatures |>
     dplyr::filter(stringr::str_detect(Fluorophore, "PBMC_Unstained"))
Zombie <- AlternateSignatures |> 
    dplyr::filter(stringr::str_detect(Fluorophore, "Zombie"))
UpdatedBeadReferences <- bind_rows(BeadSignatures, Unstained, Zombie)

UpdatedBeadReferences |> pull(Fluorophore)

 [1] "BUV395"          "BUV496"          "BUV563"          "BUV615"         
 [5] "BUV661"          "BUV737"          "BUV805"          "BV421"          
 [9] "Pacific Blue"    "BV480"           "BV510"           "BV605"          
[13] "BV650"           "BV711"           "BV750"           "BV786"          
[17] "FITC"            "Spark Blue 550"  "PerCP-Cy5.5"     "PE"             
[21] "PE-Dazzle 594"   "PE-Cy5"          "PE-Vio 770"      "APC"            
[25] "Alexa Fluor 647" "APC-R700"        "APC-Fire 750"    "APC-Fire 810"   
[29] "PBMC_Unstained"  "Zombie NIR"     

Now, just to rearrange Zombie and Unstained into the correct row order.

RelocateElements <- function(data, from, after) {
  index <- seq_len(nrow(data))
  index <- index[index != from]
  insert_at <- which(index == after)
  index <- append(index, from, after = insert_at)
  data[index, ]
}

UpdatedBeadReferences <- RelocateElements(data=UpdatedBeadReferences, from=30, after=27)
UpdatedBeadReferences |> pull(Fluorophore)

 [1] "BUV395"          "BUV496"          "BUV563"          "BUV615"         
 [5] "BUV661"          "BUV737"          "BUV805"          "BV421"          
 [9] "Pacific Blue"    "BV480"           "BV510"           "BV605"          
[13] "BV650"           "BV711"           "BV750"           "BV786"          
[17] "FITC"            "Spark Blue 550"  "PerCP-Cy5.5"     "PE"             
[21] "PE-Dazzle 594"   "PE-Cy5"          "PE-Vio 770"      "APC"            
[25] "Alexa Fluor 647" "APC-R700"        "APC-Fire 750"    "Zombie NIR"     
[29] "APC-Fire 810"    "PBMC_Unstained" 

And to make things easier, lets rename “PBMC_Unstained” to AF.

UpdatedBeadReferences <- UpdatedBeadReferences |> 
    mutate(Fluorophore=case_when(
        Fluorophore == "PBMC_Unstained" ~ "AF",
        TRUE ~ Fluorophore)
          )

tail(UpdatedBeadReferences, 5)

    Fluorophore   Antigen UV1-A UV2-A UV3-A UV4-A UV5-A UV6-A UV7-A UV8-A UV9-A
26     APC-R700    CD107a 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
27 APC-Fire 750      CD27 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
30   Zombie NIR Viability 0.001 0.003 0.002 0.003 0.004 0.005 0.009 0.007 0.007
28 APC-Fire 810      CD38 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.001 0.001
29           AF           0.046 0.087 0.075 0.095 0.146 0.251 0.537 0.405 0.372
   UV10-A UV11-A UV12-A UV13-A UV14-A UV15-A UV16-A  V1-A  V2-A  V3-A  V4-A
26  0.000  0.002  0.016  0.042  0.034  0.020  0.012 0.000 0.000 0.000 0.000
27  0.000  0.001  0.000  0.001  0.011  0.057  0.053 0.000 0.000 0.001 0.001
30  0.004  0.003  0.002  0.009  0.066  0.075  0.034 0.002 0.006 0.010 0.011
28  0.000  0.004  0.002  0.002  0.003  0.021  0.093 0.000 0.001 0.001 0.001
29  0.151  0.092  0.056  0.044  0.054  0.041  0.035 0.076 0.258 0.434 0.505
    V5-A  V6-A  V7-A  V8-A  V9-A V10-A V11-A V12-A V13-A V14-A V15-A V16-A
26 0.000 0.000 0.001 0.000 0.000 0.000 0.006 0.041 0.112 0.064 0.042 0.017
27 0.001 0.001 0.001 0.001 0.001 0.001 0.003 0.001 0.002 0.024 0.142 0.087
30 0.017 0.016 0.024 0.020 0.014 0.016 0.010 0.008 0.041 0.223 0.280 0.081
28 0.002 0.002 0.002 0.002 0.001 0.002 0.012 0.006 0.006 0.006 0.063 0.202
29 0.768 0.730 1.000 0.746 0.527 0.609 0.350 0.204 0.195 0.180 0.164 0.097
    B1-A  B2-A  B3-A  B4-A  B5-A  B6-A  B7-A  B8-A  B9-A B10-A B11-A B12-A
26 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.003 0.018 0.025 0.015 0.009
27 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.000 0.000 0.000 0.001 0.005
30 0.009 0.011 0.016 0.011 0.010 0.008 0.007 0.005 0.009 0.030 0.114 0.276
28 0.001 0.001 0.001 0.001 0.001 0.001 0.002 0.001 0.001 0.001 0.001 0.001
29 0.248 0.302 0.418 0.284 0.248 0.201 0.148 0.110 0.118 0.075 0.058 0.054
   B13-A B14-A YG1-A YG2-A YG3-A YG4-A YG5-A YG6-A YG7-A YG8-A YG9-A YG10-A
26 0.006 0.006 0.000 0.000 0.000 0.013 0.036 0.158 0.500 0.168 0.108  0.055
27 0.014 0.017 0.000 0.000 0.001 0.007 0.004 0.003 0.005 0.043 0.232  0.185
30 0.231 0.153 0.006 0.006 0.005 0.006 0.006 0.007 0.041 0.112 0.126  0.048
28 0.006 0.035 0.001 0.001 0.003 0.032 0.021 0.016 0.021 0.012 0.074  0.298
29 0.045 0.058 0.128 0.128 0.158 0.145 0.112 0.101 0.121 0.056 0.050  0.031
    R1-A  R2-A  R3-A  R4-A  R5-A  R6-A  R7-A  R8-A
26 0.028 0.121 0.640 1.000 0.700 0.374 0.349 0.155
27 0.015 0.013 0.008 0.010 0.043 0.291 1.000 0.604
30 0.009 0.012 0.030 0.151 0.595 1.000 0.970 0.316
28 0.063 0.061 0.052 0.043 0.037 0.051 0.372 1.000
29 0.039 0.049 0.051 0.054 0.031 0.028 0.028 0.017

And with that, we should be at the equivalent point where we left off.

Intermediate <- map(.x=fcs_files[1], .f=OldFashionedUnmix, 
SignatureData=UpdatedBeadReferences, returnType="fcs")
head(Intermediate[[1]], 3)

     Time    SSC-W   SSC-H     SSC-A    FSC-W   FSC-H   FSC-A  SSC-B-W SSC-B-H
1  887309 683436.8  843643  960961.0 696000.9 1473050 1708740 676320.5  608196
2 1393476 658488.6 1080396 1185714.0 665755.6 1411870 1566601 660120.8  782257
3 1258794 756646.2  714195  900654.8 723983.9 1276900 1540758 715180.6  554999
   SSC-B-A   BUV395-A     BUV496-A   BUV563-A   BUV615-A  BUV661-A   BUV737-A
1 685559.1   698.9385 142103.07416  4693.3116   40.70749  504.9595 -105.37370
2 860640.1   431.6759    494.90676 33796.1594 -837.73170  609.7961  -63.70081
3 661540.9 11175.8241    -56.28136   572.2067 1169.13066 -512.1527 -566.20791
    BUV805-A   BV421-A Pacific Blue-A   BV480-A   BV510-A   BV605-A   BV650-A
1  -288.1342 3795.2743      -394.3967 1440.6184  4549.111  947.5550 14926.849
2  1813.3706  440.2116      4273.7052  247.4025 26076.313  207.4692  4482.672
3 30539.7372 7388.5224      -623.5300  577.2415  5688.807 -678.7335 16255.307
      BV711-A   BV750-A   BV786-A     FITC-A Spark Blue 550-A PerCP-Cy5.5-A
1  97542.8745 1733.8254 -211.4355 -604.68497       45958.6701     2098.7171
2   -270.8378 1423.2353 3333.3268   68.41591         693.3397      424.2276
3 162022.4787 -770.5102 2684.0677  -97.15930       45425.0009     2690.2426
        PE-A PE-Dazzle 594-A PE-Cy5-A PE-Vio 770-A    APC-A Alexa Fluor 647-A
1 55249.1340       -515.4923 386.9981    4409.8038 1867.690         -1011.577
2   717.9945        766.2239 194.9767     762.6181 2758.993          1524.624
3   552.9612       -388.2691 357.2628    2599.5774 3812.097          2123.882
   APC-R700-A APC-Fire 750-A Zombie NIR-A APC-Fire 810-A     AF-A
1    53.23036      34909.681     707.1137       27332.27 6806.192
2  6348.41146      -2686.721    1887.3333       52433.72 4700.989
3 -2193.30755      18450.067     392.5792       29704.08 7383.022

Walk-through

Sketching a Plan

At this point in OldFashionedUnmix(), we get back our unmixed fluorophore data as a “data.frame” object. As this is handed to UnmixInternal(), other than converting it to a matrix object, we don’t really need to do any additional changes before we can put it back into the “flowFrame”’s exprs() slot.

However, as mentioned in the Journey Thus Far, since the current “flowFrame” parameter() and keyword() slots contain values originally corresponding to the “Detector” columns that are no longer present, we will need to start off by identifying and removing these.

Once these have been cleaned out, the parameter() “data” object, which contains the “$P1”, “$P10” notation for the row numbers is likely to be out of sequence for the remaining Time, SSC and FSC columns that are retained going from the raw to the unmixed .fcs files. This will require recalculating their new row number, before updating the original keyword() to reflect these number changes.

At this point, everything would be correctly formatted for the retained columns, and we can start adding in the new unmixed fluorophore columns. These would end up occupying new row entries in paramater() “data”, and taking their new “$P11”-style row number, new keyword()s would be created.

Likewise, we would then need to do an additional bit of cleanup (switching the “$SPILLOVER” matrix from one using detectors to Fluorophores, modifying filenames, etc.) before taking the updated exprs(), parameter() and keyword() slots and using new() to reassemble a new unmixed flowFrame.

At this point, this new unmixed flowFrame could be returned from UnmixInternal() to OldFashionedUnmix(), where the last conditional changes about what to call the file, and where to save/return it to can be specified.

The main thing to keep in mind, is, that when possible, we should reuse code we have already written for similar tasks during the Week 10 bonus walk-through, modifying it to the task at hand as needed. This saves us time from having to rewrite similar tasks from scratch.

UnmixInternal

As we start the handoff from OldFashionedUnmix() to our new nested function UnmixInternal(), we will be getting handed the unmixed “data.frame” object. However, we will ultimately also need the old “flowFrame” to access the existing metadata. Likewise, we will need some of the “panel” metadata from original signature matrix so that we can provide Fluorophore and Antigen information to the final .fcs file.

Skeleton

We can start with creating a function skeleton for UnmixInternal, and specify 3 arguments between the parenthesis (“ff”, “data”, “panel”). These we can then document in the roxygen2 framework as far as what the expected object type is, so that when we forget two months from now, it is easier to recall.

#' Internal function for OldFashinedUnmix, handles fixing the formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing Fluorophore and Antigen
# 
UnmixInternal <- function(ff, data, panel){
    # Code goes here
}

Building Blocks

To avoid having to work two levels down inside a nested function, we can temporatily modify OldFashionedUnmix() to return to us these 3 objects (“ff”, “data”, “panel”) via a list(), which will allow us to work with then directly in UnmixInternal(), simplifying the function writing process. After modifying the function, we need to re-run the code-block to make sure those changes are reflected in our local environment.

#'  This function unmixes our raw full-stained .fcs files using the signature
#' matrix we provide.
#' 
#' @param x A file.path to a raw full-stained .fcs file we want to unmix. 
#' @param retainThese Default "FSC|SSC|Time", used to separate out
#' columns not used for unmixing, but that should be retained for the final
#' unmixed .fcs files. 
#' @param detectorExclude Default is "-H|-W", intended to remove additional
#' detector columns other than -A, adjust as needed for your own instruments
#' configuration
#' @param SignatureData A data.frame containing a Fluorophore, Antigen and
#' Detector columns. 
#' @param returnType Default "fcs", for residual plots use "residual"
#' 
#' @importFrom flowCore read.FCS exprs
#' @importFrom dplyr select matches where
#' @importFrom utils read.csv
#' 
OldFashionedUnmix <- function(x, retainThese="FSC|SSC|Time",
 detectorExclude="-H|-W", SignatureData, returnType="fcs"){

    # Retrieve the Raw Data

    TheRawFCS <- flowCore::read.FCS(filename=x, transformation=FALSE,
     truncate_max_range = FALSE)
    TheRawMatrix <- flowCore::exprs(TheRawFCS)
    TheRawDataFrame <- data.frame(TheRawMatrix, check.names=FALSE)

    # Identify the Detector Columns

    StashedColumns <- TheRawDataFrame |> 
        dplyr::select(dplyr::matches(retainThese)) 
    WorkingColumns <- TheRawDataFrame |> 
        dplyr::select(!dplyr::matches(retainThese)) 
    WorkingColumns <- WorkingColumns |>
         dplyr::select(!dplyr::matches(detectorExclude)) 

    # Retrieve the Signature Matrix 

    if(is.data.frame(SignatureData)){
        Signatures <- SignatureData
    } else { 
        Signatures <-read.csv(SignatureData, check.names=FALSE)
    }

    # Separate Metadata from Detector Columns

    Metadata <- Signatures |> dplyr::select(!dplyr::where(is.numeric))
    Numerics <- Signatures |> dplyr::select(dplyr::where(is.numeric))

    # If not already normalized, scale the signature values

    if (any(Numerics > 1)) {
        message("Signature values greater than 1 detected, normalizing")
        n <- Numerics
        # n[n < 0] <- 0
        A <- do.call(pmax, n)
        Normalized <- n/A
        Numerics <- Normalized
    }

    # Verify your signature matrix and raw full-stained have the same number of columns

    if (!all(colnames(Numerics) == colnames(WorkingColumns))){
        stop("colnames of SignatureData due not match the internal colnames of exprs")
    }

    # Transpose both signature and full-stained matrices, and unmix with OLS

    DetectorNameBackups <- colnames(Numerics)
    TransposedSignatureValues <- t(Numerics)
    TransposedSampleValues <- t(WorkingColumns)
    LeastSquares <- lsfit(x = TransposedSignatureValues,
     y = TransposedSampleValues, intercept = FALSE)
    TransposedLeastSquares <- t(LeastSquares$coefficients)

    # Optional fork to return the residual plot instead
    if (returnType == "residuals"){
    Plot <- Hell3(LeastSquaresList = LeastSquares)
    return(Plot)
    }

    # Update the column names
    FluorophoreNames <- Metadata |> pull(Fluorophore)
    TheDetectorColNames <- colnames(WorkingColumns)
    AppendThisLetter <- sub("^[^-]*", "", TheDetectorColNames) |> unique()
    FluorophoreNames <- paste0(FluorophoreNames, AppendThisLetter)
    colnames(TransposedLeastSquares) <- FluorophoreNames

    # Bind the stashed Time, SSC and FSC columns to the new fluorophore columns
    UnmixedData <- bind_cols(StashedColumns, TransposedLeastSquares)

    # Temporary Modification
    TheList <- list(ff=TheRawFCS, data=UnmixedData, panel=Metadata)

    return(TheList)
}

At this point, we can run the function (for just the first .fcs file, as denoted by “fcs_files[1]”), escape out of the list format (via the “[[1]]”), and since the object is a “named list”, used names() to see what we are working with.

TheList <- map(.x=fcs_files[1], .f=OldFashionedUnmix,
 SignatureData=UpdatedBeadReferences, returnType="fcs")
TheList <- TheList[[1]]
names(TheList)

[1] "ff"    "data"  "panel"

With those names now identified, we can retrieve these from the list through the use of the $ accessor. Lets save them so that they have the same name as the expected arguments, making the function writing simpler to execute.

ff <- TheList$ff

data <- TheList$data

panel <- TheList$panel

We are now set to start building out UnmixInternal() in earnest.

Identifying Columns

To resituate ourselves, lets modify return() to give us back our “flowFrame”. As always, re-run the code-block to refresh the function in our local environment

#' Internal function for OldFashinedUnmix, handles fixing the
#'  formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
# 
UnmixInternal <- function(ff, data, panel){
    return(ff)
}

And then run our output line (and rinse and repeat as we continue to make modifications)

UnmixInternal(ff=ff, data=data, panel=panel)

flowFrame object 'DTR_2023_ILT_01-INF052-Ctrl_Antibody.1235515.fcs'
with 10000 cells and 74 observables:
       name   desc     range  minRange  maxRange
$P1    Time     NA   1428432         0   1428431
$P2   UV1-A     NA   4194304      -111   4194304
$P3   UV2-A     NA   4194304      -111   4194304
$P4   UV3-A     NA   4194304      -111   4194304
$P5   UV4-A     NA   4194304      -111   4194304
...     ...    ...       ...       ...       ...
$P70   R4-A     NA   4194304      -111   4194304
$P71   R5-A     NA   4194304      -111   4194304
$P72   R6-A     NA   4194304      -111   4194304
$P73   R7-A     NA   4194304      -111   4194304
$P74   R8-A     NA   4194304      -111   4194304
719 keywords are stored in the 'description' slot

Our printout of the ‘flowFrame’ is typical of what we see for raw spectral .fcs files on a 5-laser Cytek Aurora

Building off oursketching a plan notes, we can modify UnmixInternal to start off by identifying the colnames() of the original raw flowFrame

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
# 
UnmixInternal <- function(ff, data, panel){
    TheOriginalColumns <- colnames(ff)
    return(TheOriginalColumns)
}

UnmixInternal(ff=ff, data=data, panel=panel)

 [1] "Time"    "UV1-A"   "UV2-A"   "UV3-A"   "UV4-A"   "UV5-A"   "UV6-A"  
 [8] "UV7-A"   "UV8-A"   "UV9-A"   "UV10-A"  "UV11-A"  "UV12-A"  "UV13-A" 
[15] "UV14-A"  "UV15-A"  "UV16-A"  "SSC-W"   "SSC-H"   "SSC-A"   "V1-A"   
[22] "V2-A"    "V3-A"    "V4-A"    "V5-A"    "V6-A"    "V7-A"    "V8-A"   
[29] "V9-A"    "V10-A"   "V11-A"   "V12-A"   "V13-A"   "V14-A"   "V15-A"  
[36] "V16-A"   "FSC-W"   "FSC-H"   "FSC-A"   "SSC-B-W" "SSC-B-H" "SSC-B-A"
[43] "B1-A"    "B2-A"    "B3-A"    "B4-A"    "B5-A"    "B6-A"    "B7-A"   
[50] "B8-A"    "B9-A"    "B10-A"   "B11-A"   "B12-A"   "B13-A"   "B14-A"  
[57] "YG1-A"   "YG2-A"   "YG3-A"   "YG4-A"   "YG5-A"   "YG6-A"   "YG7-A"  
[64] "YG8-A"   "YG9-A"   "YG10-A"  "R1-A"    "R2-A"    "R3-A"    "R4-A"   
[71] "R5-A"    "R6-A"    "R7-A"    "R8-A"   

For the original columns, the starting column was “Time”. This was then followed by a stretch of UV laser detector columns, before encountering the “SSC” columns (“SSC” off the violet-laser in this case). Beyond, there was a stretch of Violet laser detector columns, before encountering both the “FSC” and “SSC-B” (“SSC” off the blue-laser) detectors. After this we have the detectors for the blue, yellow-green and red lasers.

The main things to note, all detector columns are denoted by only “-A” (Area) in this case, while SSC, FSC and SSC-B all have variants (“-W”, “-H”, -“A”, corresponding to Width, Height and Area). Whether these are present in your own .fcs file will be both manufacturer as well as user selected preference specific, so we need to retain some flexibility in handling situations where the defaults are not “-A”.

We can proceed to run colnames() on our unmixed “data.frame”, and modify return() so that we can see the contents when the function is run.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
# 
UnmixInternal <- function(ff, data, panel){
    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    return(TheNewColumns)
}

UnmixInternal(ff=ff, data=data, panel=panel)

 [1] "Time"              "SSC-W"             "SSC-H"            
 [4] "SSC-A"             "FSC-W"             "FSC-H"            
 [7] "FSC-A"             "SSC-B-W"           "SSC-B-H"          
[10] "SSC-B-A"           "BUV395-A"          "BUV496-A"         
[13] "BUV563-A"          "BUV615-A"          "BUV661-A"         
[16] "BUV737-A"          "BUV805-A"          "BV421-A"          
[19] "Pacific Blue-A"    "BV480-A"           "BV510-A"          
[22] "BV605-A"           "BV650-A"           "BV711-A"          
[25] "BV750-A"           "BV786-A"           "FITC-A"           
[28] "Spark Blue 550-A"  "PerCP-Cy5.5-A"     "PE-A"             
[31] "PE-Dazzle 594-A"   "PE-Cy5-A"          "PE-Vio 770-A"     
[34] "APC-A"             "Alexa Fluor 647-A" "APC-R700-A"       
[37] "APC-Fire 750-A"    "Zombie NIR-A"      "APC-Fire 810-A"   
[40] "AF-A"             

At first glance, it appears the ordering for “Time”, “SSC”, “FSC” and “SSC-B” was maintained. After these, we have the individual newly unmixed fluorophore columns.

As mentioned in our sketching a plan notes, we fortunately do not need to modify much to be able to swap “data” into the exprs() matrix slot.

However, we will need to modify parameter() and keyword() slots rather extensively. For the retained columns (“SSC”, “FSC”, etc.) all their associated values would remain the same, but the names they are stored under will need to be updated to reflect their new “$P” row number. For the new fluorophore columns, they will need to be assigned new “$P” numeric values, and the associated parameter() and keyword() components generated. And for the detector columns that were removed from exprs(), we will need to also remove their associated parameter() and keyword() entries.

To go about this, we can identify which columns were retained between our raw and unmixed data using intersect().

#' Internal function for OldFashinedUnmix, handles fixing the
#'  formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    return(RetainedColumns)
}

UnmixInternal(ff=ff, data=data, panel=panel)

 [1] "Time"    "SSC-W"   "SSC-H"   "SSC-A"   "FSC-W"   "FSC-H"   "FSC-A"  
 [8] "SSC-B-W" "SSC-B-H" "SSC-B-A"

As anticipated, we get back the “Time”, “SSC”, “FSC” and “SSC-B” columns. Next up, lets retrieve the original parameters() “data” object. Since this will use the flowCore packages paramaters() function, we add it to the roxygen2 skeleton under the “importFrom” tag for documentation, and add the package name plus “:::” before the function itself, which enables it to be correctly assigned at this point in the course (in the absence of a Namespace file, which we will cover later).

#' Internal function for OldFashinedUnmix, handles fixing the
#'  formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data
    
    return(TheOriginalParameters)
}

UnmixInternal(ff=ff, data=data, panel=panel)

        name desc   range  minRange maxRange
$P1     Time <NA> 1428432    0.0000  1428431
$P2    UV1-A <NA> 4194304 -111.0004  4194304
$P3    UV2-A <NA> 4194304 -111.0004  4194304
$P4    UV3-A <NA> 4194304 -111.0004  4194304
$P5    UV4-A <NA> 4194304 -111.0004  4194304
$P6    UV5-A <NA> 4194304 -111.0004  4194304
$P7    UV6-A <NA> 4194304 -111.0004  4194304
$P8    UV7-A <NA> 4194304 -111.0004  4194304
$P9    UV8-A <NA> 4194304 -111.0004  4194304
$P10   UV9-A <NA> 4194304 -111.0004  4194304
$P11  UV10-A <NA> 4194304 -111.0004  4194304
$P12  UV11-A <NA> 4194304 -111.0004  4194304
$P13  UV12-A <NA> 4194304 -111.0004  4194304
$P14  UV13-A <NA> 4194304 -111.0004  4194304
$P15  UV14-A <NA> 4194304 -111.0004  4194304
$P16  UV15-A <NA> 4194304 -111.0004  4194304
$P17  UV16-A <NA> 4194304 -111.0004  4194304
$P18   SSC-W <NA> 4194304    0.0000  4194303
$P19   SSC-H <NA> 4194304    0.0000  4194303
$P20   SSC-A <NA> 4194304    0.0000  4194303
$P21    V1-A <NA> 4194304 -111.0004  4194304
$P22    V2-A <NA> 4194304 -111.0004  4194304
$P23    V3-A <NA> 4194304 -111.0004  4194304
$P24    V4-A <NA> 4194304 -111.0004  4194304
$P25    V5-A <NA> 4194304 -111.0004  4194304
$P26    V6-A <NA> 4194304 -111.0004  4194304
$P27    V7-A <NA> 4194304 -111.0004  4194304
$P28    V8-A <NA> 4194304 -111.0004  4194304
$P29    V9-A <NA> 4194304 -111.0004  4194304
$P30   V10-A <NA> 4194304 -111.0004  4194304
$P31   V11-A <NA> 4194304 -111.0004  4194304
$P32   V12-A <NA> 4194304 -111.0004  4194304
$P33   V13-A <NA> 4194304 -111.0004  4194304
$P34   V14-A <NA> 4194304 -111.0004  4194304
$P35   V15-A <NA> 4194304 -111.0004  4194304
$P36   V16-A <NA> 4194304 -111.0004  4194304
$P37   FSC-W <NA> 4194304    0.0000  4194303
$P38   FSC-H <NA> 4194304    0.0000  4194303
$P39   FSC-A <NA> 4194304    0.0000  4194303
$P40 SSC-B-W <NA> 4194304    0.0000  4194303
$P41 SSC-B-H <NA> 4194304    0.0000  4194303
$P42 SSC-B-A <NA> 4194304    0.0000  4194303
$P43    B1-A <NA> 4194304 -111.0004  4194304
$P44    B2-A <NA> 4194304 -111.0004  4194304
$P45    B3-A <NA> 4194304 -111.0004  4194304
$P46    B4-A <NA> 4194304 -111.0004  4194304
$P47    B5-A <NA> 4194304 -111.0004  4194304
$P48    B6-A <NA> 4194304 -111.0004  4194304
$P49    B7-A <NA> 4194304 -111.0004  4194304
$P50    B8-A <NA> 4194304 -111.0004  4194304
$P51    B9-A <NA> 4194304 -111.0004  4194304
$P52   B10-A <NA> 4194304 -111.0004  4194304
$P53   B11-A <NA> 4194304 -111.0004  4194304
$P54   B12-A <NA> 4194304 -111.0004  4194304
$P55   B13-A <NA> 4194304 -111.0004  4194304
$P56   B14-A <NA> 4194304 -111.0004  4194304
$P57   YG1-A <NA> 4194304 -111.0004  4194304
$P58   YG2-A <NA> 4194304 -111.0004  4194304
$P59   YG3-A <NA> 4194304 -111.0004  4194304
$P60   YG4-A <NA> 4194304 -111.0004  4194304
$P61   YG5-A <NA> 4194304 -111.0004  4194304
$P62   YG6-A <NA> 4194304 -111.0004  4194304
$P63   YG7-A <NA> 4194304 -111.0004  4194304
$P64   YG8-A <NA> 4194304 -111.0004  4194304
$P65   YG9-A <NA> 4194304 -111.0004  4194304
$P66  YG10-A <NA> 4194304 -111.0004  4194304
$P67    R1-A <NA> 4194304 -111.0004  4194304
$P68    R2-A <NA> 4194304 -111.0004  4194304
$P69    R3-A <NA> 4194304 -111.0004  4194304
$P70    R4-A <NA> 4194304 -111.0004  4194304
$P71    R5-A <NA> 4194304 -111.0004  4194304
$P72    R6-A <NA> 4194304 -111.0004  4194304
$P73    R7-A <NA> 4194304 -111.0004  4194304
$P74    R8-A <NA> 4194304 -111.0004  4194304

As we can see, all the detectors are still showing as their own row entries. We can filter() the “name” column to isolate just the rows corresponding to our retained columns, so that we know based on the row.names() what keywords should be kept/renamed instead of deleted.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing 
#' Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters
#' @importFrom dplyr filter
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    
    return(KeepThese)
}

UnmixInternal(ff=ff, data=data, panel=panel)

        name desc   range minRange maxRange
$P1     Time <NA> 1428432        0  1428431
$P18   SSC-W <NA> 4194304        0  4194303
$P19   SSC-H <NA> 4194304        0  4194303
$P20   SSC-A <NA> 4194304        0  4194303
$P37   FSC-W <NA> 4194304        0  4194303
$P38   FSC-H <NA> 4194304        0  4194303
$P39   FSC-A <NA> 4194304        0  4194303
$P40 SSC-B-W <NA> 4194304        0  4194303
$P41 SSC-B-H <NA> 4194304        0  4194303
$P42 SSC-B-A <NA> 4194304        0  4194303

By modifying that line of code in adding an “!”, we can also filter() for the column names that were not retained, which allows us to target their respective keyword() entries for deletion later on.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing 
#' Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters
#' @importFrom dplyr filter
#' 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()
    
    return(GetRidThese)
}

UnmixInternal(ff=ff, data=data, panel=panel)

 [1] "$P2"  "$P3"  "$P4"  "$P5"  "$P6"  "$P7"  "$P8"  "$P9"  "$P10" "$P11"
[11] "$P12" "$P13" "$P14" "$P15" "$P16" "$P17" "$P21" "$P22" "$P23" "$P24"
[21] "$P25" "$P26" "$P27" "$P28" "$P29" "$P30" "$P31" "$P32" "$P33" "$P34"
[31] "$P35" "$P36" "$P43" "$P44" "$P45" "$P46" "$P47" "$P48" "$P49" "$P50"
[41] "$P51" "$P52" "$P53" "$P54" "$P55" "$P56" "$P57" "$P58" "$P59" "$P60"
[51] "$P61" "$P62" "$P63" "$P64" "$P65" "$P66" "$P67" "$P68" "$P69" "$P70"
[61] "$P71" "$P72" "$P73" "$P74"

Removing Excess Keywords

We are now ready to go ahead and retrieve from the raw “flowFrame” the keyword() slot, so that we can start modifying the existing description list accordingly.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing 
#' Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter
#' 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    
    return(TheOriginalDescription)
}

OriginalList <- UnmixInternal(ff=ff, data=data, panel=panel)
OriginalList[1:20]

$`$BEGINANALYSIS`
[1] "0"

$`$BEGINDATA`
[1] "49820"

$`$BEGINSTEXT`
[1] "0"

$`$BTIM`
[1] " "

$`$BYTEORD`
[1] "4,3,2,1"

$`$CYT`
[1] "Aurora"

$`$CYTOLIB_VERSION`
[1] "2.22.0"

$`$CYTSN`
[1] "U1368"

$`$DATATYPE`
[1] "F"

$`$DATE`
[1] " "

$`$ENDANALYSIS`
[1] "0"

$`$ENDDATA`
[1] "3009819"

$`$ENDSTEXT`
[1] "0"

$`$ETIM`
[1] " "

$`$FIL`
[1] "DTR_2023_ILT_01-INF052-Ctrl_Antibody.fcs"

$`$INST`
[1] "Cytekbio"

$`$MODE`
[1] "L"

$`$NEXTDATA`
[1] "0"

$`$OP`
[1] " "

$`$P10B`
[1] "32"

Within UnmixInternal(), we have the “GetRidThese” variable, which stores the “$P” row.names() values that we will be matching for (and subsequently removing) from the description list. To do this matching, lets run names() to get back the keyword names. Because of the special character nature of the dollar sign followed by variable numbers, we will need to also create a regex pattern to match for. We can do this via gsub() and paste().

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter
#' 
UnmixInternal <- function(ff, data, panel){
    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")
 
    return(RegexCombinatorial)
}

UnmixInternal(ff=ff, data=data, panel=panel)

[1] "^\\$P2($|[^0-9])|^\\$P3($|[^0-9])|^\\$P4($|[^0-9])|^\\$P5($|[^0-9])|^\\$P6($|[^0-9])|^\\$P7($|[^0-9])|^\\$P8($|[^0-9])|^\\$P9($|[^0-9])|^\\$P10($|[^0-9])|^\\$P11($|[^0-9])|^\\$P12($|[^0-9])|^\\$P13($|[^0-9])|^\\$P14($|[^0-9])|^\\$P15($|[^0-9])|^\\$P16($|[^0-9])|^\\$P17($|[^0-9])|^\\$P21($|[^0-9])|^\\$P22($|[^0-9])|^\\$P23($|[^0-9])|^\\$P24($|[^0-9])|^\\$P25($|[^0-9])|^\\$P26($|[^0-9])|^\\$P27($|[^0-9])|^\\$P28($|[^0-9])|^\\$P29($|[^0-9])|^\\$P30($|[^0-9])|^\\$P31($|[^0-9])|^\\$P32($|[^0-9])|^\\$P33($|[^0-9])|^\\$P34($|[^0-9])|^\\$P35($|[^0-9])|^\\$P36($|[^0-9])|^\\$P43($|[^0-9])|^\\$P44($|[^0-9])|^\\$P45($|[^0-9])|^\\$P46($|[^0-9])|^\\$P47($|[^0-9])|^\\$P48($|[^0-9])|^\\$P49($|[^0-9])|^\\$P50($|[^0-9])|^\\$P51($|[^0-9])|^\\$P52($|[^0-9])|^\\$P53($|[^0-9])|^\\$P54($|[^0-9])|^\\$P55($|[^0-9])|^\\$P56($|[^0-9])|^\\$P57($|[^0-9])|^\\$P58($|[^0-9])|^\\$P59($|[^0-9])|^\\$P60($|[^0-9])|^\\$P61($|[^0-9])|^\\$P62($|[^0-9])|^\\$P63($|[^0-9])|^\\$P64($|[^0-9])|^\\$P65($|[^0-9])|^\\$P66($|[^0-9])|^\\$P67($|[^0-9])|^\\$P68($|[^0-9])|^\\$P69($|[^0-9])|^\\$P70($|[^0-9])|^\\$P71($|[^0-9])|^\\$P72($|[^0-9])|^\\$P73($|[^0-9])|^\\$P74($|[^0-9])"

At this point, using grepl() to pattern match, we can subset out the matches using the base R [] approach to get back a vector of keyword names from the list that need to be excluded.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter
#'  
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
 
    return(IdentifiedElimination)
}

UnmixInternal(ff=ff, data=data, panel=panel)

  [1] "$P10B"    "$P10E"    "$P10N"    "$P10R"    "$P10TYPE" "$P10V"   
  [7] "$P11B"    "$P11E"    "$P11N"    "$P11R"    "$P11TYPE" "$P11V"   
 [13] "$P12B"    "$P12E"    "$P12N"    "$P12R"    "$P12TYPE" "$P12V"   
 [19] "$P13B"    "$P13E"    "$P13N"    "$P13R"    "$P13TYPE" "$P13V"   
 [25] "$P14B"    "$P14E"    "$P14N"    "$P14R"    "$P14TYPE" "$P14V"   
 [31] "$P15B"    "$P15E"    "$P15N"    "$P15R"    "$P15TYPE" "$P15V"   
 [37] "$P16B"    "$P16E"    "$P16N"    "$P16R"    "$P16TYPE" "$P16V"   
 [43] "$P17B"    "$P17E"    "$P17N"    "$P17R"    "$P17TYPE" "$P17V"   
 [49] "$P21B"    "$P21E"    "$P21N"    "$P21R"    "$P21TYPE" "$P21V"   
 [55] "$P22B"    "$P22E"    "$P22N"    "$P22R"    "$P22TYPE" "$P22V"   
 [61] "$P23B"    "$P23E"    "$P23N"    "$P23R"    "$P23TYPE" "$P23V"   
 [67] "$P24B"    "$P24E"    "$P24N"    "$P24R"    "$P24TYPE" "$P24V"   
 [73] "$P25B"    "$P25E"    "$P25N"    "$P25R"    "$P25TYPE" "$P25V"   
 [79] "$P26B"    "$P26E"    "$P26N"    "$P26R"    "$P26TYPE" "$P26V"   
 [85] "$P27B"    "$P27E"    "$P27N"    "$P27R"    "$P27TYPE" "$P27V"   
 [91] "$P28B"    "$P28E"    "$P28N"    "$P28R"    "$P28TYPE" "$P28V"   
 [97] "$P29B"    "$P29E"    "$P29N"    "$P29R"    "$P29TYPE" "$P29V"   
[103] "$P2B"     "$P2E"     "$P2N"     "$P2R"     "$P2TYPE"  "$P2V"    
[109] "$P30B"    "$P30E"    "$P30N"    "$P30R"    "$P30TYPE" "$P30V"   
[115] "$P31B"    "$P31E"    "$P31N"    "$P31R"    "$P31TYPE" "$P31V"   
[121] "$P32B"    "$P32E"    "$P32N"    "$P32R"    "$P32TYPE" "$P32V"   
[127] "$P33B"    "$P33E"    "$P33N"    "$P33R"    "$P33TYPE" "$P33V"   
[133] "$P34B"    "$P34E"    "$P34N"    "$P34R"    "$P34TYPE" "$P34V"   
[139] "$P35B"    "$P35E"    "$P35N"    "$P35R"    "$P35TYPE" "$P35V"   
[145] "$P36B"    "$P36E"    "$P36N"    "$P36R"    "$P36TYPE" "$P36V"   
[151] "$P3B"     "$P3E"     "$P3N"     "$P3R"     "$P3TYPE"  "$P3V"    
[157] "$P43B"    "$P43E"    "$P43N"    "$P43R"    "$P43TYPE" "$P43V"   
[163] "$P44B"    "$P44E"    "$P44N"    "$P44R"    "$P44TYPE" "$P44V"   
[169] "$P45B"    "$P45E"    "$P45N"    "$P45R"    "$P45TYPE" "$P45V"   
[175] "$P46B"    "$P46E"    "$P46N"    "$P46R"    "$P46TYPE" "$P46V"   
[181] "$P47B"    "$P47E"    "$P47N"    "$P47R"    "$P47TYPE" "$P47V"   
[187] "$P48B"    "$P48E"    "$P48N"    "$P48R"    "$P48TYPE" "$P48V"   
[193] "$P49B"    "$P49E"    "$P49N"    "$P49R"    "$P49TYPE" "$P49V"   
[199] "$P4B"     "$P4E"     "$P4N"     "$P4R"     "$P4TYPE"  "$P4V"    
[205] "$P50B"    "$P50E"    "$P50N"    "$P50R"    "$P50TYPE" "$P50V"   
[211] "$P51B"    "$P51E"    "$P51N"    "$P51R"    "$P51TYPE" "$P51V"   
[217] "$P52B"    "$P52E"    "$P52N"    "$P52R"    "$P52TYPE" "$P52V"   
[223] "$P53B"    "$P53E"    "$P53N"    "$P53R"    "$P53TYPE" "$P53V"   
[229] "$P54B"    "$P54E"    "$P54N"    "$P54R"    "$P54TYPE" "$P54V"   
[235] "$P55B"    "$P55E"    "$P55N"    "$P55R"    "$P55TYPE" "$P55V"   
[241] "$P56B"    "$P56E"    "$P56N"    "$P56R"    "$P56TYPE" "$P56V"   
[247] "$P57B"    "$P57E"    "$P57N"    "$P57R"    "$P57TYPE" "$P57V"   
[253] "$P58B"    "$P58E"    "$P58N"    "$P58R"    "$P58TYPE" "$P58V"   
[259] "$P59B"    "$P59E"    "$P59N"    "$P59R"    "$P59TYPE" "$P59V"   
[265] "$P5B"     "$P5E"     "$P5N"     "$P5R"     "$P5TYPE"  "$P5V"    
[271] "$P60B"    "$P60E"    "$P60N"    "$P60R"    "$P60TYPE" "$P60V"   
[277] "$P61B"    "$P61E"    "$P61N"    "$P61R"    "$P61TYPE" "$P61V"   
[283] "$P62B"    "$P62E"    "$P62N"    "$P62R"    "$P62TYPE" "$P62V"   
[289] "$P63B"    "$P63E"    "$P63N"    "$P63R"    "$P63TYPE" "$P63V"   
[295] "$P64B"    "$P64E"    "$P64N"    "$P64R"    "$P64TYPE" "$P64V"   
[301] "$P65B"    "$P65E"    "$P65N"    "$P65R"    "$P65TYPE" "$P65V"   
[307] "$P66B"    "$P66E"    "$P66N"    "$P66R"    "$P66TYPE" "$P66V"   
[313] "$P67B"    "$P67E"    "$P67N"    "$P67R"    "$P67TYPE" "$P67V"   
[319] "$P68B"    "$P68E"    "$P68N"    "$P68R"    "$P68TYPE" "$P68V"   
[325] "$P69B"    "$P69E"    "$P69N"    "$P69R"    "$P69TYPE" "$P69V"   
[331] "$P6B"     "$P6E"     "$P6N"     "$P6R"     "$P6TYPE"  "$P6V"    
[337] "$P70B"    "$P70E"    "$P70N"    "$P70R"    "$P70TYPE" "$P70V"   
[343] "$P71B"    "$P71E"    "$P71N"    "$P71R"    "$P71TYPE" "$P71V"   
[349] "$P72B"    "$P72E"    "$P72N"    "$P72R"    "$P72TYPE" "$P72V"   
[355] "$P73B"    "$P73E"    "$P73N"    "$P73R"    "$P73TYPE" "$P73V"   
[361] "$P74B"    "$P74E"    "$P74N"    "$P74R"    "$P74TYPE" "$P74V"   
[367] "$P7B"     "$P7E"     "$P7N"     "$P7R"     "$P7TYPE"  "$P7V"    
[373] "$P8B"     "$P8E"     "$P8N"     "$P8R"     "$P8TYPE"  "$P8V"    
[379] "$P9B"     "$P9E"     "$P9N"     "$P9R"     "$P9TYPE"  "$P9V"    

Likewise, with a well-placed “!” we can also denote those being retained.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing 
#' Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter
#' 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]
 
    return(IdentifiedRetention)
}

UnmixInternal(ff=ff, data=data, panel=panel)

  [1] "$BEGINANALYSIS"     "$BEGINDATA"         "$BEGINSTEXT"       
  [4] "$BTIM"              "$BYTEORD"           "$CYT"              
  [7] "$CYTOLIB_VERSION"   "$CYTSN"             "$DATATYPE"         
 [10] "$DATE"              "$ENDANALYSIS"       "$ENDDATA"          
 [13] "$ENDSTEXT"          "$ETIM"              "$FIL"              
 [16] "$INST"              "$MODE"              "$NEXTDATA"         
 [19] "$OP"                "$P18B"              "$P18E"             
 [22] "$P18N"              "$P18R"              "$P18TYPE"          
 [25] "$P18V"              "$P19B"              "$P19E"             
 [28] "$P19N"              "$P19R"              "$P19TYPE"          
 [31] "$P19V"              "$P1B"               "$P1E"              
 [34] "$P1N"               "$P1R"               "$P1TYPE"           
 [37] "$P20B"              "$P20E"              "$P20N"             
 [40] "$P20R"              "$P20TYPE"           "$P20V"             
 [43] "$P37B"              "$P37E"              "$P37N"             
 [46] "$P37R"              "$P37TYPE"           "$P37V"             
 [49] "$P38B"              "$P38E"              "$P38N"             
 [52] "$P38R"              "$P38TYPE"           "$P38V"             
 [55] "$P39B"              "$P39E"              "$P39N"             
 [58] "$P39R"              "$P39TYPE"           "$P39V"             
 [61] "$P40B"              "$P40E"              "$P40N"             
 [64] "$P40R"              "$P40TYPE"           "$P40V"             
 [67] "$P41B"              "$P41E"              "$P41N"             
 [70] "$P41R"              "$P41TYPE"           "$P41V"             
 [73] "$P42B"              "$P42E"              "$P42N"             
 [76] "$P42R"              "$P42TYPE"           "$P42V"             
 [79] "$PAR"               "$PROJ"              "$SPILLOVER"        
 [82] "$TIMESTEP"          "$TOT"               "$VOL"              
 [85] "APPLY COMPENSATION" "CHARSET"            "CREATOR"           
 [88] "FCSversion"         "FILENAME"           "flowCore_$P10Rmax" 
 [91] "flowCore_$P10Rmin"  "flowCore_$P11Rmax"  "flowCore_$P11Rmin" 
 [94] "flowCore_$P12Rmax"  "flowCore_$P12Rmin"  "flowCore_$P13Rmax" 
 [97] "flowCore_$P13Rmin"  "flowCore_$P14Rmax"  "flowCore_$P14Rmin" 
[100] "flowCore_$P15Rmax"  "flowCore_$P15Rmin"  "flowCore_$P16Rmax" 
[103] "flowCore_$P16Rmin"  "flowCore_$P17Rmax"  "flowCore_$P17Rmin" 
[106] "flowCore_$P18Rmax"  "flowCore_$P18Rmin"  "flowCore_$P19Rmax" 
[109] "flowCore_$P19Rmin"  "flowCore_$P1Rmax"   "flowCore_$P1Rmin"  
[112] "flowCore_$P20Rmax"  "flowCore_$P20Rmin"  "flowCore_$P21Rmax" 
[115] "flowCore_$P21Rmin"  "flowCore_$P22Rmax"  "flowCore_$P22Rmin" 
[118] "flowCore_$P23Rmax"  "flowCore_$P23Rmin"  "flowCore_$P24Rmax" 
[121] "flowCore_$P24Rmin"  "flowCore_$P25Rmax"  "flowCore_$P25Rmin" 
[124] "flowCore_$P26Rmax"  "flowCore_$P26Rmin"  "flowCore_$P27Rmax" 
[127] "flowCore_$P27Rmin"  "flowCore_$P28Rmax"  "flowCore_$P28Rmin" 
[130] "flowCore_$P29Rmax"  "flowCore_$P29Rmin"  "flowCore_$P2Rmax"  
[133] "flowCore_$P2Rmin"   "flowCore_$P30Rmax"  "flowCore_$P30Rmin" 
[136] "flowCore_$P31Rmax"  "flowCore_$P31Rmin"  "flowCore_$P32Rmax" 
[139] "flowCore_$P32Rmin"  "flowCore_$P33Rmax"  "flowCore_$P33Rmin" 
[142] "flowCore_$P34Rmax"  "flowCore_$P34Rmin"  "flowCore_$P35Rmax" 
[145] "flowCore_$P35Rmin"  "flowCore_$P36Rmax"  "flowCore_$P36Rmin" 
[148] "flowCore_$P37Rmax"  "flowCore_$P37Rmin"  "flowCore_$P38Rmax" 
[151] "flowCore_$P38Rmin"  "flowCore_$P39Rmax"  "flowCore_$P39Rmin" 
[154] "flowCore_$P3Rmax"   "flowCore_$P3Rmin"   "flowCore_$P40Rmax" 
[157] "flowCore_$P40Rmin"  "flowCore_$P41Rmax"  "flowCore_$P41Rmin" 
[160] "flowCore_$P42Rmax"  "flowCore_$P42Rmin"  "flowCore_$P43Rmax" 
[163] "flowCore_$P43Rmin"  "flowCore_$P44Rmax"  "flowCore_$P44Rmin" 
[166] "flowCore_$P45Rmax"  "flowCore_$P45Rmin"  "flowCore_$P46Rmax" 
[169] "flowCore_$P46Rmin"  "flowCore_$P47Rmax"  "flowCore_$P47Rmin" 
[172] "flowCore_$P48Rmax"  "flowCore_$P48Rmin"  "flowCore_$P49Rmax" 
[175] "flowCore_$P49Rmin"  "flowCore_$P4Rmax"   "flowCore_$P4Rmin"  
[178] "flowCore_$P50Rmax"  "flowCore_$P50Rmin"  "flowCore_$P51Rmax" 
[181] "flowCore_$P51Rmin"  "flowCore_$P52Rmax"  "flowCore_$P52Rmin" 
[184] "flowCore_$P53Rmax"  "flowCore_$P53Rmin"  "flowCore_$P54Rmax" 
[187] "flowCore_$P54Rmin"  "flowCore_$P55Rmax"  "flowCore_$P55Rmin" 
[190] "flowCore_$P56Rmax"  "flowCore_$P56Rmin"  "flowCore_$P57Rmax" 
[193] "flowCore_$P57Rmin"  "flowCore_$P58Rmax"  "flowCore_$P58Rmin" 
[196] "flowCore_$P59Rmax"  "flowCore_$P59Rmin"  "flowCore_$P5Rmax"  
[199] "flowCore_$P5Rmin"   "flowCore_$P60Rmax"  "flowCore_$P60Rmin" 
[202] "flowCore_$P61Rmax"  "flowCore_$P61Rmin"  "flowCore_$P62Rmax" 
[205] "flowCore_$P62Rmin"  "flowCore_$P63Rmax"  "flowCore_$P63Rmin" 
[208] "flowCore_$P64Rmax"  "flowCore_$P64Rmin"  "flowCore_$P65Rmax" 
[211] "flowCore_$P65Rmin"  "flowCore_$P66Rmax"  "flowCore_$P66Rmin" 
[214] "flowCore_$P67Rmax"  "flowCore_$P67Rmin"  "flowCore_$P68Rmax" 
[217] "flowCore_$P68Rmin"  "flowCore_$P69Rmax"  "flowCore_$P69Rmin" 
[220] "flowCore_$P6Rmax"   "flowCore_$P6Rmin"   "flowCore_$P70Rmax" 
[223] "flowCore_$P70Rmin"  "flowCore_$P71Rmax"  "flowCore_$P71Rmin" 
[226] "flowCore_$P72Rmax"  "flowCore_$P72Rmin"  "flowCore_$P73Rmax" 
[229] "flowCore_$P73Rmin"  "flowCore_$P74Rmax"  "flowCore_$P74Rmin" 
[232] "flowCore_$P7Rmax"   "flowCore_$P7Rmin"   "flowCore_$P8Rmax"  
[235] "flowCore_$P8Rmin"   "flowCore_$P9Rmax"   "flowCore_$P9Rmin"  
[238] "FSC ASF"            "GROUPNAME"          "GUID"              
[241] "LASER1ASF"          "LASER1DELAY"        "LASER1NAME"        
[244] "LASER2ASF"          "LASER2DELAY"        "LASER2NAME"        
[247] "LASER3ASF"          "LASER3DELAY"        "LASER3NAME"        
[250] "LASER4ASF"          "LASER4DELAY"        "LASER4NAME"        
[253] "LASER5ASF"          "LASER5DELAY"        "LASER5NAME"        
[256] "P10DISPLAY"         "P11DISPLAY"         "P12DISPLAY"        
[259] "P13DISPLAY"         "P14DISPLAY"         "P15DISPLAY"        
[262] "P16DISPLAY"         "P17DISPLAY"         "P18DISPLAY"        
[265] "P19DISPLAY"         "P1DISPLAY"          "P20DISPLAY"        
[268] "P21DISPLAY"         "P22DISPLAY"         "P23DISPLAY"        
[271] "P24DISPLAY"         "P25DISPLAY"         "P26DISPLAY"        
[274] "P27DISPLAY"         "P28DISPLAY"         "P29DISPLAY"        
[277] "P2DISPLAY"          "P30DISPLAY"         "P31DISPLAY"        
[280] "P32DISPLAY"         "P33DISPLAY"         "P34DISPLAY"        
[283] "P35DISPLAY"         "P36DISPLAY"         "P37DISPLAY"        
[286] "P38DISPLAY"         "P39DISPLAY"         "P3DISPLAY"         
[289] "P40DISPLAY"         "P41DISPLAY"         "P42DISPLAY"        
[292] "P43DISPLAY"         "P44DISPLAY"         "P45DISPLAY"        
[295] "P46DISPLAY"         "P47DISPLAY"         "P48DISPLAY"        
[298] "P49DISPLAY"         "P4DISPLAY"          "P50DISPLAY"        
[301] "P51DISPLAY"         "P52DISPLAY"         "P53DISPLAY"        
[304] "P54DISPLAY"         "P55DISPLAY"         "P56DISPLAY"        
[307] "P57DISPLAY"         "P58DISPLAY"         "P59DISPLAY"        
[310] "P5DISPLAY"          "P60DISPLAY"         "P61DISPLAY"        
[313] "P62DISPLAY"         "P63DISPLAY"         "P64DISPLAY"        
[316] "P65DISPLAY"         "P66DISPLAY"         "P67DISPLAY"        
[319] "P68DISPLAY"         "P69DISPLAY"         "P6DISPLAY"         
[322] "P70DISPLAY"         "P71DISPLAY"         "P72DISPLAY"        
[325] "P73DISPLAY"         "P74DISPLAY"         "P7DISPLAY"         
[328] "P8DISPLAY"          "P9DISPLAY"          "THRESHOLD"         
[331] "transformation"     "TUBENAME"           "USERSETTINGNAME"   
[334] "WINDOW EXTENSION"   "ORIGINALGUID"      

Looking at the respective vectors of names to exclude/retain, we can see that the keywords containing “$P” in their names have been correctly designated. However, we stll have keyword names corresponding to the “flowCore_” and “PDISPLAY” varieties that contain equivalent rowname “$P72” or “P72” values that didn’t pattern match, and were not appropiately handled.

Rather than try to write an all-encompassing argument (and end up in the 7th circle of RegEx hell), we can instead run our intermediate retrieved vector through subsequent pattern matching rounds for these sequentially. This will ensure that cleanup is carried out correctly, and also provide ability to troubleshoot more effectively if needed.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing 
#' Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter
#' 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
 
    return(SecondRetention)
}

UnmixInternal(ff=ff, data=data, panel=panel)

  [1] "$BEGINANALYSIS"     "$BEGINDATA"         "$BEGINSTEXT"       
  [4] "$BTIM"              "$BYTEORD"           "$CYT"              
  [7] "$CYTOLIB_VERSION"   "$CYTSN"             "$DATATYPE"         
 [10] "$DATE"              "$ENDANALYSIS"       "$ENDDATA"          
 [13] "$ENDSTEXT"          "$ETIM"              "$FIL"              
 [16] "$INST"              "$MODE"              "$NEXTDATA"         
 [19] "$OP"                "$P18B"              "$P18E"             
 [22] "$P18N"              "$P18R"              "$P18TYPE"          
 [25] "$P18V"              "$P19B"              "$P19E"             
 [28] "$P19N"              "$P19R"              "$P19TYPE"          
 [31] "$P19V"              "$P1B"               "$P1E"              
 [34] "$P1N"               "$P1R"               "$P1TYPE"           
 [37] "$P20B"              "$P20E"              "$P20N"             
 [40] "$P20R"              "$P20TYPE"           "$P20V"             
 [43] "$P37B"              "$P37E"              "$P37N"             
 [46] "$P37R"              "$P37TYPE"           "$P37V"             
 [49] "$P38B"              "$P38E"              "$P38N"             
 [52] "$P38R"              "$P38TYPE"           "$P38V"             
 [55] "$P39B"              "$P39E"              "$P39N"             
 [58] "$P39R"              "$P39TYPE"           "$P39V"             
 [61] "$P40B"              "$P40E"              "$P40N"             
 [64] "$P40R"              "$P40TYPE"           "$P40V"             
 [67] "$P41B"              "$P41E"              "$P41N"             
 [70] "$P41R"              "$P41TYPE"           "$P41V"             
 [73] "$P42B"              "$P42E"              "$P42N"             
 [76] "$P42R"              "$P42TYPE"           "$P42V"             
 [79] "$PAR"               "$PROJ"              "$SPILLOVER"        
 [82] "$TIMESTEP"          "$TOT"               "$VOL"              
 [85] "APPLY COMPENSATION" "CHARSET"            "CREATOR"           
 [88] "FCSversion"         "FILENAME"           "flowCore_$P18Rmax" 
 [91] "flowCore_$P18Rmin"  "flowCore_$P19Rmax"  "flowCore_$P19Rmin" 
 [94] "flowCore_$P1Rmax"   "flowCore_$P1Rmin"   "flowCore_$P20Rmax" 
 [97] "flowCore_$P20Rmin"  "flowCore_$P37Rmax"  "flowCore_$P37Rmin" 
[100] "flowCore_$P38Rmax"  "flowCore_$P38Rmin"  "flowCore_$P39Rmax" 
[103] "flowCore_$P39Rmin"  "flowCore_$P40Rmax"  "flowCore_$P40Rmin" 
[106] "flowCore_$P41Rmax"  "flowCore_$P41Rmin"  "flowCore_$P42Rmax" 
[109] "flowCore_$P42Rmin"  "FSC ASF"            "GROUPNAME"         
[112] "GUID"               "LASER1ASF"          "LASER1DELAY"       
[115] "LASER1NAME"         "LASER2ASF"          "LASER2DELAY"       
[118] "LASER2NAME"         "LASER3ASF"          "LASER3DELAY"       
[121] "LASER3NAME"         "LASER4ASF"          "LASER4DELAY"       
[124] "LASER4NAME"         "LASER5ASF"          "LASER5DELAY"       
[127] "LASER5NAME"         "P10DISPLAY"         "P11DISPLAY"        
[130] "P12DISPLAY"         "P13DISPLAY"         "P14DISPLAY"        
[133] "P15DISPLAY"         "P16DISPLAY"         "P17DISPLAY"        
[136] "P18DISPLAY"         "P19DISPLAY"         "P1DISPLAY"         
[139] "P20DISPLAY"         "P21DISPLAY"         "P22DISPLAY"        
[142] "P23DISPLAY"         "P24DISPLAY"         "P25DISPLAY"        
[145] "P26DISPLAY"         "P27DISPLAY"         "P28DISPLAY"        
[148] "P29DISPLAY"         "P2DISPLAY"          "P30DISPLAY"        
[151] "P31DISPLAY"         "P32DISPLAY"         "P33DISPLAY"        
[154] "P34DISPLAY"         "P35DISPLAY"         "P36DISPLAY"        
[157] "P37DISPLAY"         "P38DISPLAY"         "P39DISPLAY"        
[160] "P3DISPLAY"          "P40DISPLAY"         "P41DISPLAY"        
[163] "P42DISPLAY"         "P43DISPLAY"         "P44DISPLAY"        
[166] "P45DISPLAY"         "P46DISPLAY"         "P47DISPLAY"        
[169] "P48DISPLAY"         "P49DISPLAY"         "P4DISPLAY"         
[172] "P50DISPLAY"         "P51DISPLAY"         "P52DISPLAY"        
[175] "P53DISPLAY"         "P54DISPLAY"         "P55DISPLAY"        
[178] "P56DISPLAY"         "P57DISPLAY"         "P58DISPLAY"        
[181] "P59DISPLAY"         "P5DISPLAY"          "P60DISPLAY"        
[184] "P61DISPLAY"         "P62DISPLAY"         "P63DISPLAY"        
[187] "P64DISPLAY"         "P65DISPLAY"         "P66DISPLAY"        
[190] "P67DISPLAY"         "P68DISPLAY"         "P69DISPLAY"        
[193] "P6DISPLAY"          "P70DISPLAY"         "P71DISPLAY"        
[196] "P72DISPLAY"         "P73DISPLAY"         "P74DISPLAY"        
[199] "P7DISPLAY"          "P8DISPLAY"          "P9DISPLAY"         
[202] "THRESHOLD"          "transformation"     "TUBENAME"          
[205] "USERSETTINGNAME"    "WINDOW EXTENSION"   "ORIGINALGUID"      

The extra “flowCore_$P” keywords leftover from the original “Detector” columns have now been removed. Now, we just need to repeat the process for the “PDisplay” style keywords.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing 
#' Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter
#' 
UnmixInternal <- function(ff, data, panel){

     # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    return(ThirdRetention)
}

UnmixInternal(ff=ff, data=data, panel=panel)

  [1] "$BEGINANALYSIS"     "$BEGINDATA"         "$BEGINSTEXT"       
  [4] "$BTIM"              "$BYTEORD"           "$CYT"              
  [7] "$CYTOLIB_VERSION"   "$CYTSN"             "$DATATYPE"         
 [10] "$DATE"              "$ENDANALYSIS"       "$ENDDATA"          
 [13] "$ENDSTEXT"          "$ETIM"              "$FIL"              
 [16] "$INST"              "$MODE"              "$NEXTDATA"         
 [19] "$OP"                "$P18B"              "$P18E"             
 [22] "$P18N"              "$P18R"              "$P18TYPE"          
 [25] "$P18V"              "$P19B"              "$P19E"             
 [28] "$P19N"              "$P19R"              "$P19TYPE"          
 [31] "$P19V"              "$P1B"               "$P1E"              
 [34] "$P1N"               "$P1R"               "$P1TYPE"           
 [37] "$P20B"              "$P20E"              "$P20N"             
 [40] "$P20R"              "$P20TYPE"           "$P20V"             
 [43] "$P37B"              "$P37E"              "$P37N"             
 [46] "$P37R"              "$P37TYPE"           "$P37V"             
 [49] "$P38B"              "$P38E"              "$P38N"             
 [52] "$P38R"              "$P38TYPE"           "$P38V"             
 [55] "$P39B"              "$P39E"              "$P39N"             
 [58] "$P39R"              "$P39TYPE"           "$P39V"             
 [61] "$P40B"              "$P40E"              "$P40N"             
 [64] "$P40R"              "$P40TYPE"           "$P40V"             
 [67] "$P41B"              "$P41E"              "$P41N"             
 [70] "$P41R"              "$P41TYPE"           "$P41V"             
 [73] "$P42B"              "$P42E"              "$P42N"             
 [76] "$P42R"              "$P42TYPE"           "$P42V"             
 [79] "$PAR"               "$PROJ"              "$SPILLOVER"        
 [82] "$TIMESTEP"          "$TOT"               "$VOL"              
 [85] "APPLY COMPENSATION" "CHARSET"            "CREATOR"           
 [88] "FCSversion"         "FILENAME"           "flowCore_$P18Rmax" 
 [91] "flowCore_$P18Rmin"  "flowCore_$P19Rmax"  "flowCore_$P19Rmin" 
 [94] "flowCore_$P1Rmax"   "flowCore_$P1Rmin"   "flowCore_$P20Rmax" 
 [97] "flowCore_$P20Rmin"  "flowCore_$P37Rmax"  "flowCore_$P37Rmin" 
[100] "flowCore_$P38Rmax"  "flowCore_$P38Rmin"  "flowCore_$P39Rmax" 
[103] "flowCore_$P39Rmin"  "flowCore_$P40Rmax"  "flowCore_$P40Rmin" 
[106] "flowCore_$P41Rmax"  "flowCore_$P41Rmin"  "flowCore_$P42Rmax" 
[109] "flowCore_$P42Rmin"  "FSC ASF"            "GROUPNAME"         
[112] "GUID"               "LASER1ASF"          "LASER1DELAY"       
[115] "LASER1NAME"         "LASER2ASF"          "LASER2DELAY"       
[118] "LASER2NAME"         "LASER3ASF"          "LASER3DELAY"       
[121] "LASER3NAME"         "LASER4ASF"          "LASER4DELAY"       
[124] "LASER4NAME"         "LASER5ASF"          "LASER5DELAY"       
[127] "LASER5NAME"         "P18DISPLAY"         "P19DISPLAY"        
[130] "P1DISPLAY"          "P20DISPLAY"         "P37DISPLAY"        
[133] "P38DISPLAY"         "P39DISPLAY"         "P40DISPLAY"        
[136] "P41DISPLAY"         "P42DISPLAY"         "THRESHOLD"         
[139] "transformation"     "TUBENAME"           "USERSETTINGNAME"   
[142] "WINDOW EXTENSION"   "ORIGINALGUID"      

And with this, we are left with a vector of names containing just the keywords we are interested in retaining. We can go ahead and subset these out from the original description list using the [] approach.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter
#' 
UnmixInternal <- function(ff, data, panel){

     # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    return(IntermediateDescription)
}

DescriptionIntermediate <- UnmixInternal(ff=ff, data=data, panel=panel)
DescriptionIntermediate[20:30]

$`$P18B`
[1] "32"

$`$P18E`
[1] "0,0"

$`$P18N`
[1] "SSC-W"

$`$P18R`
[1] "4194304"

$`$P18TYPE`
[1] "Side_Scatter"

$`$P18V`
[1] "337"

$`$P19B`
[1] "32"

$`$P19E`
[1] "0,0"

$`$P19N`
[1] "SSC-H"

$`$P19R`
[1] "4194304"

$`$P19TYPE`
[1] "Side_Scatter"

Renaming retained keywords

With the “detector” parameter() and keyword() entries now removed, we still need to renumber and rename the retained “SSC”, “FSC” and “SSC-B” entries that were retained going from a raw to unmixed .fcs file. By renumbering, we ensure everything is sequential, and that the new unmixed fluorophore entries don’t accidentally overwrite our “FSC” entries due to old numbering.

Renumbering parameters data

We can start using tibble packages rownames_to_column() function to shift the rownames() containing the current “$P” to their own column (“OriginalRowNumber”). We can then pipe through to dplyr’s mutate() function, and identifying what the new “NewRowNumber” value would be through the combination of paste0() and row_number() functions. We can then relocate() “NewRowNumber” to the first column position. We update the roxygen2 skeleton and add appropiate “::” entries as we go.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate
#' @importFrom tibble rownames_to_column
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    return(IntermediateParameters)
}

UnmixInternal(ff=ff, data=data, panel=panel)

   NewRowNumber OriginalRowNumber    name desc   range minRange maxRange
1           $P1               $P1    Time <NA> 1428432        0  1428431
2           $P2              $P18   SSC-W <NA> 4194304        0  4194303
3           $P3              $P19   SSC-H <NA> 4194304        0  4194303
4           $P4              $P20   SSC-A <NA> 4194304        0  4194303
5           $P5              $P37   FSC-W <NA> 4194304        0  4194303
6           $P6              $P38   FSC-H <NA> 4194304        0  4194303
7           $P7              $P39   FSC-A <NA> 4194304        0  4194303
8           $P8              $P40 SSC-B-W <NA> 4194304        0  4194303
9           $P9              $P41 SSC-B-H <NA> 4194304        0  4194303
10         $P10              $P42 SSC-B-A <NA> 4194304        0  4194303

Having generated the new “$P” rownumbers, we will need need to match for the “OriginalRowNumber” entries is the keyword() description list, and swap in the new rownumber name without changing the underlying data. Since there are two moving pieces (old rownumber, new row number), and they are actively modifying names in a single list, this is more suited for the use of a “for-loop”.

We can first pull() both the old and new row numbers and save them to their own vectors. To avoid accidents while coding out the for-loop, lets also duplicate the current list.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' @importFrom tibble rownames_to_column
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    return(NewRowNumbers)
}

UnmixInternal(ff=ff, data=data, panel=panel)

 [1] "$P1"  "$P2"  "$P3"  "$P4"  "$P5"  "$P6"  "$P7"  "$P8"  "$P9"  "$P10"

Unlike other cases of iteration we have seen with map2(), for-loops typically iterate through a single vector. We can work around this by having the for-loop iterate through index positions using the seq_along() function which identifies the length of our vector. For these retrieved index positions (“i”), we can then inside the for-loop subset the vectors for NewRowNumbers and OldRowNumbers via the [i] approach. We can set up an equivalent value without the “$P” component for later use.

Since this can be complicated, we can run an output print() line to check that everything is working as expected at this stage.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' @importFrom tibble rownames_to_column
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        print(paste("NewX is", NewX, ", OldX is", OldX))

    }

    return(NewRowNumbers)
}

UnmixInternal(ff=ff, data=data, panel=panel)

[1] "NewX is $P1 , OldX is $P1"
[1] "NewX is $P2 , OldX is $P18"
[1] "NewX is $P3 , OldX is $P19"
[1] "NewX is $P4 , OldX is $P20"
[1] "NewX is $P5 , OldX is $P37"
[1] "NewX is $P6 , OldX is $P38"
[1] "NewX is $P7 , OldX is $P39"
[1] "NewX is $P8 , OldX is $P40"
[1] "NewX is $P9 , OldX is $P41"
[1] "NewX is $P10 , OldX is $P42"

 [1] "$P1"  "$P2"  "$P3"  "$P4"  "$P5"  "$P6"  "$P7"  "$P8"  "$P9"  "$P10"

From our output to the console window, we can see the for-loop is grabbing the New and Old Row Number values as we were expecting it to. Now it is time to check the names in our list, and match any that have the corresponding values present (similar to how we did our eliminate/retain matching earlier). We can take advantage of our previous code, and set it with the goal of renaming the matches instead of removing them.

We can do this by first setting up a “RegEx” pattern via paste0() and gsub(), before matching using retrieved names() through the use of [] and grepl() pattern matching (with the perl argument set to TRUE)

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' @importFrom tibble rownames_to_column
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

    }

    return(InternalIdentifiedRename)
}

UnmixInternal(ff=ff, data=data, panel=panel)

[1] "$P42B"             "$P42E"             "$P42N"            
[4] "$P42R"             "$P42TYPE"          "$P42V"            
[7] "flowCore_$P42Rmax" "flowCore_$P42Rmin"

Looking at the last output line, the for-loop identified both the “$P” and “flowCore_$P” keywords that matched the old row number “$P”. Now that these keywords are identified, we can use a combination of names(), match() and “[]” to identify the “$P” portion, which will be overwritten by the new value held within “InternalRenamed”

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' @importFrom tibble rownames_to_column
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed
    }

    return(names(ForLoopDescription))
}

UnmixInternal(ff=ff, data=data, panel=panel)

  [1] "$BEGINANALYSIS"     "$BEGINDATA"         "$BEGINSTEXT"       
  [4] "$BTIM"              "$BYTEORD"           "$CYT"              
  [7] "$CYTOLIB_VERSION"   "$CYTSN"             "$DATATYPE"         
 [10] "$DATE"              "$ENDANALYSIS"       "$ENDDATA"          
 [13] "$ENDSTEXT"          "$ETIM"              "$FIL"              
 [16] "$INST"              "$MODE"              "$NEXTDATA"         
 [19] "$OP"                "$P2B"               "$P2E"              
 [22] "$P2N"               "$P2R"               "$P2TYPE"           
 [25] "$P2V"               "$P3B"               "$P3E"              
 [28] "$P3N"               "$P3R"               "$P3TYPE"           
 [31] "$P3V"               "$P1B"               "$P1E"              
 [34] "$P1N"               "$P1R"               "$P1TYPE"           
 [37] "$P4B"               "$P4E"               "$P4N"              
 [40] "$P4R"               "$P4TYPE"            "$P4V"              
 [43] "$P5B"               "$P5E"               "$P5N"              
 [46] "$P5R"               "$P5TYPE"            "$P5V"              
 [49] "$P6B"               "$P6E"               "$P6N"              
 [52] "$P6R"               "$P6TYPE"            "$P6V"              
 [55] "$P7B"               "$P7E"               "$P7N"              
 [58] "$P7R"               "$P7TYPE"            "$P7V"              
 [61] "$P8B"               "$P8E"               "$P8N"              
 [64] "$P8R"               "$P8TYPE"            "$P8V"              
 [67] "$P9B"               "$P9E"               "$P9N"              
 [70] "$P9R"               "$P9TYPE"            "$P9V"              
 [73] "$P10B"              "$P10E"              "$P10N"             
 [76] "$P10R"              "$P10TYPE"           "$P10V"             
 [79] "$PAR"               "$PROJ"              "$SPILLOVER"        
 [82] "$TIMESTEP"          "$TOT"               "$VOL"              
 [85] "APPLY COMPENSATION" "CHARSET"            "CREATOR"           
 [88] "FCSversion"         "FILENAME"           "flowCore_$P2Rmax"  
 [91] "flowCore_$P2Rmin"   "flowCore_$P3Rmax"   "flowCore_$P3Rmin"  
 [94] "flowCore_$P1Rmax"   "flowCore_$P1Rmin"   "flowCore_$P4Rmax"  
 [97] "flowCore_$P4Rmin"   "flowCore_$P5Rmax"   "flowCore_$P5Rmin"  
[100] "flowCore_$P6Rmax"   "flowCore_$P6Rmin"   "flowCore_$P7Rmax"  
[103] "flowCore_$P7Rmin"   "flowCore_$P8Rmax"   "flowCore_$P8Rmin"  
[106] "flowCore_$P9Rmax"   "flowCore_$P9Rmin"   "flowCore_$P10Rmax" 
[109] "flowCore_$P10Rmin"  "FSC ASF"            "GROUPNAME"         
[112] "GUID"               "LASER1ASF"          "LASER1DELAY"       
[115] "LASER1NAME"         "LASER2ASF"          "LASER2DELAY"       
[118] "LASER2NAME"         "LASER3ASF"          "LASER3DELAY"       
[121] "LASER3NAME"         "LASER4ASF"          "LASER4DELAY"       
[124] "LASER4NAME"         "LASER5ASF"          "LASER5DELAY"       
[127] "LASER5NAME"         "P18DISPLAY"         "P19DISPLAY"        
[130] "P1DISPLAY"          "P20DISPLAY"         "P37DISPLAY"        
[133] "P38DISPLAY"         "P39DISPLAY"         "P40DISPLAY"        
[136] "P41DISPLAY"         "P42DISPLAY"         "THRESHOLD"         
[139] "transformation"     "TUBENAME"           "USERSETTINGNAME"   
[142] "WINDOW EXTENSION"   "ORIGINALGUID"      

From our returned names(), the retained “$P” and “flowCore” style keywords have had their original row number values updated with the new ones. This leaves only the “PDisplay” style ones left to rename. We can modify our previous code example to try to similar match the existing keywords with the old rownumber values in this format.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' @importFrom tibble rownames_to_column
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

    }

    return(ThirdRename)
}

UnmixInternal(ff=ff, data=data, panel=panel)

[1] "P42DISPLAY"

With the respective keywords correctly identified, we can extend the same names(), match() and “[]” strategy to replace the old row number with the corresponding new value.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' @importFrom tibble rownames_to_column
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

        # Final renaming in action for "PDisplay"

        DisplayRenamePattern <- paste0("^(\\$)?P", OldXNum, "(.*)$")
        ThirdRenamed <- sub(DisplayRenamePattern, 
        paste0("\\1P", NewXNum, "\\2"), ThirdRename)

        names(ForLoopDescription)[match(ThirdRename,
         names(ForLoopDescription))] <- ThirdRenamed
    }

    return(names(ForLoopDescription))
}

UnmixInternal(ff=ff, data=data, panel=panel)

  [1] "$BEGINANALYSIS"     "$BEGINDATA"         "$BEGINSTEXT"       
  [4] "$BTIM"              "$BYTEORD"           "$CYT"              
  [7] "$CYTOLIB_VERSION"   "$CYTSN"             "$DATATYPE"         
 [10] "$DATE"              "$ENDANALYSIS"       "$ENDDATA"          
 [13] "$ENDSTEXT"          "$ETIM"              "$FIL"              
 [16] "$INST"              "$MODE"              "$NEXTDATA"         
 [19] "$OP"                "$P2B"               "$P2E"              
 [22] "$P2N"               "$P2R"               "$P2TYPE"           
 [25] "$P2V"               "$P3B"               "$P3E"              
 [28] "$P3N"               "$P3R"               "$P3TYPE"           
 [31] "$P3V"               "$P1B"               "$P1E"              
 [34] "$P1N"               "$P1R"               "$P1TYPE"           
 [37] "$P4B"               "$P4E"               "$P4N"              
 [40] "$P4R"               "$P4TYPE"            "$P4V"              
 [43] "$P5B"               "$P5E"               "$P5N"              
 [46] "$P5R"               "$P5TYPE"            "$P5V"              
 [49] "$P6B"               "$P6E"               "$P6N"              
 [52] "$P6R"               "$P6TYPE"            "$P6V"              
 [55] "$P7B"               "$P7E"               "$P7N"              
 [58] "$P7R"               "$P7TYPE"            "$P7V"              
 [61] "$P8B"               "$P8E"               "$P8N"              
 [64] "$P8R"               "$P8TYPE"            "$P8V"              
 [67] "$P9B"               "$P9E"               "$P9N"              
 [70] "$P9R"               "$P9TYPE"            "$P9V"              
 [73] "$P10B"              "$P10E"              "$P10N"             
 [76] "$P10R"              "$P10TYPE"           "$P10V"             
 [79] "$PAR"               "$PROJ"              "$SPILLOVER"        
 [82] "$TIMESTEP"          "$TOT"               "$VOL"              
 [85] "APPLY COMPENSATION" "CHARSET"            "CREATOR"           
 [88] "FCSversion"         "FILENAME"           "flowCore_$P2Rmax"  
 [91] "flowCore_$P2Rmin"   "flowCore_$P3Rmax"   "flowCore_$P3Rmin"  
 [94] "flowCore_$P1Rmax"   "flowCore_$P1Rmin"   "flowCore_$P4Rmax"  
 [97] "flowCore_$P4Rmin"   "flowCore_$P5Rmax"   "flowCore_$P5Rmin"  
[100] "flowCore_$P6Rmax"   "flowCore_$P6Rmin"   "flowCore_$P7Rmax"  
[103] "flowCore_$P7Rmin"   "flowCore_$P8Rmax"   "flowCore_$P8Rmin"  
[106] "flowCore_$P9Rmax"   "flowCore_$P9Rmin"   "flowCore_$P10Rmax" 
[109] "flowCore_$P10Rmin"  "FSC ASF"            "GROUPNAME"         
[112] "GUID"               "LASER1ASF"          "LASER1DELAY"       
[115] "LASER1NAME"         "LASER2ASF"          "LASER2DELAY"       
[118] "LASER2NAME"         "LASER3ASF"          "LASER3DELAY"       
[121] "LASER3NAME"         "LASER4ASF"          "LASER4DELAY"       
[124] "LASER4NAME"         "LASER5ASF"          "LASER5DELAY"       
[127] "LASER5NAME"         "P2DISPLAY"          "P3DISPLAY"         
[130] "P1DISPLAY"          "P4DISPLAY"          "P5DISPLAY"         
[133] "P6DISPLAY"          "P7DISPLAY"          "P8DISPLAY"         
[136] "P9DISPLAY"          "P10DISPLAY"         "THRESHOLD"         
[139] "transformation"     "TUBENAME"           "USERSETTINGNAME"   
[142] "WINDOW EXTENSION"   "ORIGINALGUID"      

And with that, woohoo! We have cleaned out the keyword() description list of all the old “Detector” associated keywords, and updated the retained ones to reflect the new row number that is tied to their parameters() entry.

Adding Fluorophores to Parameters.

We can now switch gears and tackle adding new row entries to the parameters() “data”, corresponding to each of the new fluorophore columns that can be found in our unmixed data.frame.

Since we no longer need to keep the old row number entries, lets remove them from the data.frame using select() and return the new row numbers to rownames() location using the tibble() packages column_to_rownames() function.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' select
#' @importFrom tibble rownames_to_column column_to_rownames
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

        # Final renaming in action for "PDisplay"

        DisplayRenamePattern <- paste0("^(\\$)?P", OldXNum, "(.*)$")
        ThirdRenamed <- sub(DisplayRenamePattern, 
        paste0("\\1P", NewXNum, "\\2"), ThirdRename)

        names(ForLoopDescription)[match(ThirdRename,
         names(ForLoopDescription))] <- ThirdRenamed
    }

    # Updating Parameters with the new row names
    IntermediateParameters <- IntermediateParameters |>
         dplyr::select(-OriginalRowNumber) |>
         tibble::column_to_rownames("NewRowNumber")

    return(IntermediateParameters)
}

UnmixInternal(ff=ff, data=data, panel=panel)

        name desc   range minRange maxRange
$P1     Time <NA> 1428432        0  1428431
$P2    SSC-W <NA> 4194304        0  4194303
$P3    SSC-H <NA> 4194304        0  4194303
$P4    SSC-A <NA> 4194304        0  4194303
$P5    FSC-W <NA> 4194304        0  4194303
$P6    FSC-H <NA> 4194304        0  4194303
$P7    FSC-A <NA> 4194304        0  4194303
$P8  SSC-B-W <NA> 4194304        0  4194303
$P9  SSC-B-H <NA> 4194304        0  4194303
$P10 SSC-B-A <NA> 4194304        0  4194303

At this point, we just need to add the new fluorophore entries as new columns. If you completed the Week 10 bonus walk-through, you may remember that the ParameterUpdate() function roughly took over at this point in adding new row entries for the metadata columns that we are adding in as part of Concatenate(). While we are trying to add “Fluorophore” rows, we can pull this previous function to refresh our memory of how it worked.

#' Internal for Concatenate, creates the new parameter
#' data rows needed to properly integrate new keyword columns
#' in exprs matrix
#' 
#' @param flowFrame A flowframe object (source of the
#'  original parameters that will be modified)
#' @param NewColumns A matrix containing the new keyword columns
#' that will be appended to the exprs matrix, that need
#' to be represented in parameters as new row entries.
#' 
#' @importFrom Biobase pData
#' @importFrom flowCore parameters
#'  
ParameterUpdate <- function(flowFrame, NewColumns){
    NewColumnLength <- ncol(NewColumns)
    NewColumnNames <- colnames(NewColumns)
    OldParameters <- Biobase::pData(flowCore::parameters(flowFrame))
    NewParameter <- max(as.integer(gsub("\\$P", "", rownames(OldParameters)))) + 1
    NewParameter <- seq(NewParameter, length.out = NewColumnLength)
    NewParameter <- paste0("$P", NewParameter)
    
    UpdatedParameters <- do.call(rbind,  lapply(NewColumnNames, function(i){
                        vec <- NewColumns[,i]
                        rg <- range(vec)
                        data.frame(name = i,
                       desc = NA,
                       range = diff(rg) + 1,
                       minRange = rg[1],
                       maxRange = rg[2])
                    }))
          
    rownames(UpdatedParameters) <- NewParameter
    return(UpdatedParameters)
}

In the case ParameterUpdate(), it looks like it was calculating from scratch the range entries that were placed within the parameters() data. Given the variable number of entries that would be associated with individual metadata columns, this made sense in that context.

Let’s go ahead and duplicate, and rename it to UnmixedParameterUpdate(), in this case taking two arguments, the OldParameters (with modified names) and the unmixed data.

#' Internal for UnmixInternal, creates the new parameter
#' data rows needed to properly integrate new fluorophore columns
#' in exprs matrix
#' 
#' @param OldParameters The parameter data.frame with 
#' modified row numbers
#' @param NewExprs The unmixed data that will eventually 
#' be placed back into exprs
#' 
#' @importFrom Biobase pData
#'  
UnmixedParameterUpdate <- function(OldParameters, NewExprs){
    NewColumnLength <- ncol(NewExprs)
    NewColumnNames <- colnames(NewExprs)
    NewParameter <- max(as.integer(gsub("\\$P", "", rownames(OldParameters)))) + 1
    NewParameter <- seq(NewParameter, length.out = NewColumnLength)
    NewParameter <- paste0("$P", NewParameter)
    
    UpdatedParameters <- do.call(rbind,  lapply(NewColumnNames, function(i){
                        vec <- NewExprs[,i]
                        rg <- range(vec)
                        data.frame(name = i,
                       desc = NA,
                       range = diff(rg) + 1,
                       minRange = rg[1],
                       maxRange = rg[2])
                    }))
          
    rownames(UpdatedParameters) <- NewParameter
    return(UpdatedParameters)
}

We can then run UnmixedParameterUpdate() as the next line in UnmixInternal(), allowing us to reuse the existing testing code line to see if our modifications work or not.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' select
#' @importFrom tibble rownames_to_column column_to_rownames
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

        # Final renaming in action for "PDisplay"

        DisplayRenamePattern <- paste0("^(\\$)?P", OldXNum, "(.*)$")
        ThirdRenamed <- sub(DisplayRenamePattern, 
        paste0("\\1P", NewXNum, "\\2"), ThirdRename)

        names(ForLoopDescription)[match(ThirdRename,
         names(ForLoopDescription))] <- ThirdRenamed
    }

    # Updating Parameters with the new row names
    IntermediateParameters <- IntermediateParameters |>
         dplyr::select(-OriginalRowNumber) |>
         tibble::column_to_rownames("NewRowNumber")

    # Add the new parameter rows for the Unmixed Fluorophores

    UpdatedParameters <- UnmixedParameterUpdate(OldParameters=IntermediateParameters, NewExprs=data)

    return(UpdatedParameters)
}

UnmixInternal(ff=ff, data=data, panel=panel)

                  name desc      range    minRange   maxRange
$P11              Time   NA 1427255.00      47.000 1427301.00
$P12             SSC-W   NA  961201.75  603867.250 1565068.00
$P13             SSC-H   NA 2466889.00  382992.000 2849880.00
$P14             SSC-A   NA 2930507.12  487546.125 3418052.25
$P15             FSC-W   NA  298928.69  631033.500  929961.19
$P16             FSC-H   NA 1378613.00  771388.000 2150000.00
$P17             FSC-A   NA 1360593.75 1130293.500 2490886.25
$P18           SSC-B-W   NA  854714.75  609400.125 1464113.88
$P19           SSC-B-H   NA 2300011.00  236794.000 2536804.00
$P20           SSC-B-A   NA 2571360.00  304509.000 2875868.00
$P21          BUV395-A   NA  354765.16   -2969.387  351794.78
$P22          BUV496-A   NA  301708.56   -5376.999  296330.56
$P23          BUV563-A   NA  123051.59   -1880.933  121169.65
$P24          BUV615-A   NA   23600.31   -2759.261   20840.05
$P25          BUV661-A   NA  102924.35   -2444.321  100479.02
$P26          BUV737-A   NA   29700.56   -4261.511   25438.05
$P27          BUV805-A   NA   85887.39   -3231.964   82654.42
$P28           BV421-A   NA   73622.39   -1516.054   72105.34
$P29    Pacific Blue-A   NA  126358.92   -6703.583  119654.34
$P30           BV480-A   NA  134737.55   -4603.185  130133.37
$P31           BV510-A   NA  245198.04  -10436.561  234760.48
$P32           BV605-A   NA  267329.00   -4852.530  262475.47
$P33           BV650-A   NA   95771.71   -2332.773   93437.94
$P34           BV711-A   NA  568725.94   -2820.329  565904.61
$P35           BV750-A   NA   64794.47   -5470.791   59322.68
$P36           BV786-A   NA   92285.52   -4786.773   87497.75
$P37            FITC-A   NA   21211.55   -6360.410   14850.14
$P38  Spark Blue 550-A   NA   94648.33   -3787.336   90859.99
$P39     PerCP-Cy5.5-A   NA   24755.70   -2849.255   21905.44
$P40              PE-A   NA  171530.16   -6650.227  164878.94
$P41   PE-Dazzle 594-A   NA   82786.40   -7177.538   75607.86
$P42          PE-Cy5-A   NA  150033.66  -15412.636  134620.03
$P43      PE-Vio 770-A   NA  161959.71   -1189.624  160769.09
$P44             APC-A   NA  401504.87   -4901.897  396601.97
$P45 Alexa Fluor 647-A   NA   64410.11  -12397.356   52011.76
$P46        APC-R700-A   NA  215018.96  -13436.218  201581.75
$P47    APC-Fire 750-A   NA  117511.27  -20948.627   96561.64
$P48      Zombie NIR-A   NA  299799.75   -4611.892  295186.86
$P49    APC-Fire 810-A   NA  181083.33   -1863.998  179218.34
$P50              AF-A   NA   54109.65  -15178.174   38930.47

While we get back the fluorophore rows, our rownames() start off at “$P11”. So close, but looks like we will need to modify UnmixedParameterUpdate() to first exclude retained column names (Time through SSC-B) from consideration.

#' Internal for UnmixInternal, creates the new parameter
#' data rows needed to properly integrate new fluorophore columns
#' in exprs matrix
#' 
#' @param OldParameters The parameter data.frame with 
#' modified row numbers
#' @param NewExprs The unmixed data that will eventually 
#' be placed back into exprs
#' 
#' @importFrom Biobase pData
#' @importFrom dplyr pull select
#' @importFrom tidyselect all_of
#'  
UnmixedParameterUpdate <- function(OldParameters, NewExprs){

    # Remove the retained
    OldNames <- OldParameters |> dplyr::pull(name) |> unname()
    Overlapped <- intersect(OldNames, colnames(NewExprs))
    NewExprs <- NewExprs |> 
        dplyr::select(-tidyselect::all_of(Overlapped))

    # Create new rows for the unmixed fluorophore columns

    NewColumnLength <- ncol(NewExprs)
    NewColumnNames <- colnames(NewExprs)
    NewParameter <- max(as.integer(
        gsub("\\$P", "", rownames(OldParameters)))) + 1
    NewParameter <- seq(NewParameter,
     length.out = NewColumnLength)
    NewParameter <- paste0("$P", NewParameter)
    
    UpdatedParameters <- do.call(rbind, lapply(NewColumnNames, function(i){
                        vec <- NewExprs[,i]
                        rg <- range(vec)
                        data.frame(name = i,
                       desc = NA,
                       range = diff(rg) + 1,
                       minRange = rg[1],
                       maxRange = rg[2])
                    }))
          
    rownames(UpdatedParameters) <- NewParameter
    return(UpdatedParameters)
}

After re-running the code block to refresh the function in our local environment, lets try the output line of code again.

UnmixInternal(ff=ff, data=data, panel=panel)

                  name desc     range   minRange  maxRange
$P11          BUV395-A   NA 354765.16  -2969.387 351794.78
$P12          BUV496-A   NA 301708.56  -5376.999 296330.56
$P13          BUV563-A   NA 123051.59  -1880.933 121169.65
$P14          BUV615-A   NA  23600.31  -2759.261  20840.05
$P15          BUV661-A   NA 102924.35  -2444.321 100479.02
$P16          BUV737-A   NA  29700.56  -4261.511  25438.05
$P17          BUV805-A   NA  85887.39  -3231.964  82654.42
$P18           BV421-A   NA  73622.39  -1516.054  72105.34
$P19    Pacific Blue-A   NA 126358.92  -6703.583 119654.34
$P20           BV480-A   NA 134737.55  -4603.185 130133.37
$P21           BV510-A   NA 245198.04 -10436.561 234760.48
$P22           BV605-A   NA 267329.00  -4852.530 262475.47
$P23           BV650-A   NA  95771.71  -2332.773  93437.94
$P24           BV711-A   NA 568725.94  -2820.329 565904.61
$P25           BV750-A   NA  64794.47  -5470.791  59322.68
$P26           BV786-A   NA  92285.52  -4786.773  87497.75
$P27            FITC-A   NA  21211.55  -6360.410  14850.14
$P28  Spark Blue 550-A   NA  94648.33  -3787.336  90859.99
$P29     PerCP-Cy5.5-A   NA  24755.70  -2849.255  21905.44
$P30              PE-A   NA 171530.16  -6650.227 164878.94
$P31   PE-Dazzle 594-A   NA  82786.40  -7177.538  75607.86
$P32          PE-Cy5-A   NA 150033.66 -15412.636 134620.03
$P33      PE-Vio 770-A   NA 161959.71  -1189.624 160769.09
$P34             APC-A   NA 401504.87  -4901.897 396601.97
$P35 Alexa Fluor 647-A   NA  64410.11 -12397.356  52011.76
$P36        APC-R700-A   NA 215018.96 -13436.218 201581.75
$P37    APC-Fire 750-A   NA 117511.27 -20948.627  96561.64
$P38      Zombie NIR-A   NA 299799.75  -4611.892 295186.86
$P39    APC-Fire 810-A   NA 181083.33  -1863.998 179218.34
$P40              AF-A   NA  54109.65 -15178.174  38930.47

In this case, the retained are no longer included, so the first fluorophore is listed at the expected starting row number (“$P11”, right after the last “SSC-B” column in “OldParameters”).

Comparison to Instrument Unmixed

Before we go too far, lets make sure whether the range, minRange and maxRange values are going to be comparable to those present in the unmixed .fcs file if we had unmixed on the Cytek Aurora using SpectroFlo. Let’s write a file.path() to the data folder from Week 08, which had unmixed .fcs file acquired and unmixed on the same instrument as the ones we are working with today (just for a different panel).

UnmixedFiles <- file.path("course", "08_WaysToGate", "data")
UnmixedFCS <- list.files(UnmixedFiles, pattern=".fcs", full.names=TRUE)
UnmixedExample <- flowCore::read.FCS(UnmixedFCS[1])
UnmixedExample

flowCore::parameters(UnmixedExample)@data

flowCore::keyword(UnmixedExample)[260:310]

Looking at these examples, we can see that for the unmixed fluorophores, we get a different pattern for range, minRange, and maxRange, as all the values are kind of default in their appearance.

Parsing what we can see, range appears to be the same for both the scatters and the fluorophores (4194304). minRange however is lower for the Fluorophores (-111.0001), as is the maxRange (4192505.7500). While our approach to dynamically calculate these worked well for the metadata columns, lets not rock the boat too much on this unmixing attempt (since we want to be able to take these files back into commercial software if we want, and don’t want unexpected bugs on the visualization). Since we are trying to faithfully replicate for now, lets adjust those internal assignments so that we don’t calculate out range, minRange and maxRange, but instead append the expected values.

#' Internal for UnmixInternal, creates the new parameter
#' data rows needed to properly integrate new fluorophore columns
#' in exprs matrix
#' 
#' @param OldParameters The parameter data.frame with 
#' modified row numbers
#' @param NewExprs The unmixed data that will eventually 
#' be placed back into exprs
#' 
#' @importFrom Biobase pData
#' @importFrom dplyr pull select
#' @importFrom tidyselect all_of
#'  
UnmixedParameterUpdate <- function(OldParameters, NewExprs){

    # Remove the Retaind
    OldNames <- OldParameters |> dplyr::pull(name) |> unname()
    Overlapped <- intersect(OldNames, colnames(NewExprs))
    NewExprs <- NewExprs |> select(-all_of(Overlapped))

    # Create new rows for the unmixed fluorophore columns

    NewColumnLength <- ncol(NewExprs)
    NewColumnNames <- colnames(NewExprs)
    NewParameter <- max(as.integer(gsub("\\$P", "", rownames(OldParameters)))) + 1
    NewParameter <- seq(NewParameter, length.out = NewColumnLength)
    NewParameter <- paste0("$P", NewParameter)

    # Provide Hard Coded Numbers for Range, minRange and maxRange
    SSCRange <- OldParameters[2,3] # Hard-Coded based on Cytek Aurora SpectroFlo unmixed value
    MinRange <- -111.0001 # Hard-Coded based on Cytek Aurora SpectroFlo unmixed value
    MaxRange <- 4192505.7500 # Hard-Coded based on Cytek Aurora SpectroFlo unmixed value
    
    UpdatedParameters <- do.call(rbind,  lapply(NewColumnNames, function(i){
                        vec <- NewExprs[,i]
                        rg <- range(vec)
                        data.frame(name = i,
                       desc = NA,
                       range = SSCRange,
                       minRange = MinRange,
                       maxRange = MaxRange)
                    }))
          
    rownames(UpdatedParameters) <- NewParameter
    return(UpdatedParameters)
}

UnmixInternal(ff=ff, data=data, panel=panel)

                  name desc   range  minRange maxRange
$P11          BUV395-A   NA 4194304 -111.0001  4192506
$P12          BUV496-A   NA 4194304 -111.0001  4192506
$P13          BUV563-A   NA 4194304 -111.0001  4192506
$P14          BUV615-A   NA 4194304 -111.0001  4192506
$P15          BUV661-A   NA 4194304 -111.0001  4192506
$P16          BUV737-A   NA 4194304 -111.0001  4192506
$P17          BUV805-A   NA 4194304 -111.0001  4192506
$P18           BV421-A   NA 4194304 -111.0001  4192506
$P19    Pacific Blue-A   NA 4194304 -111.0001  4192506
$P20           BV480-A   NA 4194304 -111.0001  4192506
$P21           BV510-A   NA 4194304 -111.0001  4192506
$P22           BV605-A   NA 4194304 -111.0001  4192506
$P23           BV650-A   NA 4194304 -111.0001  4192506
$P24           BV711-A   NA 4194304 -111.0001  4192506
$P25           BV750-A   NA 4194304 -111.0001  4192506
$P26           BV786-A   NA 4194304 -111.0001  4192506
$P27            FITC-A   NA 4194304 -111.0001  4192506
$P28  Spark Blue 550-A   NA 4194304 -111.0001  4192506
$P29     PerCP-Cy5.5-A   NA 4194304 -111.0001  4192506
$P30              PE-A   NA 4194304 -111.0001  4192506
$P31   PE-Dazzle 594-A   NA 4194304 -111.0001  4192506
$P32          PE-Cy5-A   NA 4194304 -111.0001  4192506
$P33      PE-Vio 770-A   NA 4194304 -111.0001  4192506
$P34             APC-A   NA 4194304 -111.0001  4192506
$P35 Alexa Fluor 647-A   NA 4194304 -111.0001  4192506
$P36        APC-R700-A   NA 4194304 -111.0001  4192506
$P37    APC-Fire 750-A   NA 4194304 -111.0001  4192506
$P38      Zombie NIR-A   NA 4194304 -111.0001  4192506
$P39    APC-Fire 810-A   NA 4194304 -111.0001  4192506
$P40              AF-A   NA 4194304 -111.0001  4192506

Close enough for this attempt. One thing to note, for “desc”, we don’t have any of the antigen names listed yet. We can get these corresponding values from the third “panel” argument for UnmixInternal, which so far we have specified and not yet used. Let’s go ahead and change that.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' select
#' @importFrom tibble rownames_to_column column_to_rownames
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

        # Final renaming in action for "PDisplay"

        DisplayRenamePattern <- paste0("^(\\$)?P", OldXNum, "(.*)$")
        ThirdRenamed <- sub(DisplayRenamePattern, 
        paste0("\\1P", NewXNum, "\\2"), ThirdRename)

        names(ForLoopDescription)[match(ThirdRename,
         names(ForLoopDescription))] <- ThirdRenamed
    }

    # Updating Parameters with the new row names
    IntermediateParameters <- IntermediateParameters |>
         dplyr::select(-OriginalRowNumber) |>
         tibble::column_to_rownames("NewRowNumber")

    # Add the new parameter rows for the Unmixed Fluorophores

    UpdatedParameters <- UnmixedParameterUpdate(
        OldParameters=IntermediateParameters, NewExprs=data)

    TheAntigens <- panel |> dplyr::pull(Antigen)
    UpdatedParameters$desc <- TheAntigens

    return(UpdatedParameters)
}

UnmixInternal(ff=ff, data=data, panel=panel)

                  name      desc   range  minRange maxRange
$P11          BUV395-A     CD62L 4194304 -111.0001  4192506
$P12          BUV496-A       CD8 4194304 -111.0001  4192506
$P13          BUV563-A      CD69 4194304 -111.0001  4192506
$P14          BUV615-A      CCR4 4194304 -111.0001  4192506
$P15          BUV661-A       VD2 4194304 -111.0001  4192506
$P16          BUV737-A     CXCR3 4194304 -111.0001  4192506
$P17          BUV805-A       CD4 4194304 -111.0001  4192506
$P18           BV421-A     CD127 4194304 -111.0001  4192506
$P19    Pacific Blue-A Dump_CD14 4194304 -111.0001  4192506
$P20           BV480-A     CD161 4194304 -111.0001  4192506
$P21           BV510-A    CD45RA 4194304 -111.0001  4192506
$P22           BV605-A      CD56 4194304 -111.0001  4192506
$P23           BV650-A      CCR7 4194304 -111.0001  4192506
$P24           BV711-A       CD7 4194304 -111.0001  4192506
$P25           BV750-A      IFNg 4194304 -111.0001  4192506
$P26           BV786-A      CCR6 4194304 -111.0001  4192506
$P27            FITC-A  Va24Ja18 4194304 -111.0001  4192506
$P28  Spark Blue 550-A       CD3 4194304 -111.0001  4192506
$P29     PerCP-Cy5.5-A      CD26 4194304 -111.0001  4192506
$P30              PE-A     NKG2D 4194304 -111.0001  4192506
$P31   PE-Dazzle 594-A      TNFa 4194304 -111.0001  4192506
$P32          PE-Cy5-A      CD25 4194304 -111.0001  4192506
$P33      PE-Vio 770-A       PD1 4194304 -111.0001  4192506
$P34             APC-A      CD16 4194304 -111.0001  4192506
$P35 Alexa Fluor 647-A     Va7.2 4194304 -111.0001  4192506
$P36        APC-R700-A    CD107a 4194304 -111.0001  4192506
$P37    APC-Fire 750-A      CD27 4194304 -111.0001  4192506
$P38      Zombie NIR-A Viability 4194304 -111.0001  4192506
$P39    APC-Fire 810-A      CD38 4194304 -111.0001  4192506
$P40              AF-A           4194304 -111.0001  4192506

Alright, the new parameter() data rows corresponding to the unmixed fluorophores have been generated, which we can bind with the old parameters (containing “Time”, “SSC”, “FSC”, and “SSC-B”) using either rbind() or bind_rows(). We can now move forward and for each row generate out the corresponding sets of keyword() that will go into the description list. For Week 10 when writing Concatenate(), we handled this through the use of a for-loop that would create these in sequence for each “$P” rownumber provided. We can copy this existing code over, and modify it so that the outputs match what is expected for the unmixed .fcs file.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' select
#' @importFrom tibble rownames_to_column column_to_rownames
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

        # Final renaming in action for "PDisplay"

        DisplayRenamePattern <- paste0("^(\\$)?P", OldXNum, "(.*)$")
        ThirdRenamed <- sub(DisplayRenamePattern, 
        paste0("\\1P", NewXNum, "\\2"), ThirdRename)

        names(ForLoopDescription)[match(ThirdRename,
         names(ForLoopDescription))] <- ThirdRenamed
    }

    # Updating Parameters with the new row names
    IntermediateParameters <- IntermediateParameters |>
         dplyr::select(-OriginalRowNumber) |>
         tibble::column_to_rownames("NewRowNumber")

    # Add the new parameter rows for the Unmixed Fluorophores
    UpdatedParameters <- UnmixedParameterUpdate(
        OldParameters=IntermediateParameters, NewExprs=data)

    TheAntigens <- panel |> pull(Antigen)
    UpdatedParameters$desc <- TheAntigens
    pd <- rbind(IntermediateParameters, UpdatedParameters)

    # Generate new keywords for the new fluorophore rows in parameters

    new_pid <- rownames(UpdatedParameters)
    new_kw <- ForLoopDescription

    for (i in new_pid){
        NoDollarCode <- sub("^\\$", "", i) # For "PDisplay keyword"
        new_kw[paste0(i,"B")] <- new_kw["$P1B"] # Bytes
        new_kw[paste0(i,"E")] <- "0,0"
        new_kw[paste0(i,"N")] <- pd[[i,1]] # Fluorophore Name
        new_kw[paste0(i,"R")] <- pd[[i,5]] # Range Default
        new_kw[paste0(i,"S")] <- pd[[i,2]] # Antigen Name
        new_kw[paste0(i,"TYPE")] <- "Unmixed_Fluorescence"
        new_kw[paste0(i,"V")] <- "0" # Voltage/Gain, unmixed default 0
        new_kw[paste0("flowCore_", i,"Rmax")] <- pd[[i,5]] # maxRange
        new_kw[paste0("flowCore_", i,"Rmin")] <- pd[[i,4]] # minRange
        new_kw[paste0(NoDollarCode,"DISPLAY")] <- "LOG"
    }

    return(names(new_kw))
}

UnmixInternal(ff=ff, data=data, panel=panel)

  [1] "$BEGINANALYSIS"     "$BEGINDATA"         "$BEGINSTEXT"       
  [4] "$BTIM"              "$BYTEORD"           "$CYT"              
  [7] "$CYTOLIB_VERSION"   "$CYTSN"             "$DATATYPE"         
 [10] "$DATE"              "$ENDANALYSIS"       "$ENDDATA"          
 [13] "$ENDSTEXT"          "$ETIM"              "$FIL"              
 [16] "$INST"              "$MODE"              "$NEXTDATA"         
 [19] "$OP"                "$P2B"               "$P2E"              
 [22] "$P2N"               "$P2R"               "$P2TYPE"           
 [25] "$P2V"               "$P3B"               "$P3E"              
 [28] "$P3N"               "$P3R"               "$P3TYPE"           
 [31] "$P3V"               "$P1B"               "$P1E"              
 [34] "$P1N"               "$P1R"               "$P1TYPE"           
 [37] "$P4B"               "$P4E"               "$P4N"              
 [40] "$P4R"               "$P4TYPE"            "$P4V"              
 [43] "$P5B"               "$P5E"               "$P5N"              
 [46] "$P5R"               "$P5TYPE"            "$P5V"              
 [49] "$P6B"               "$P6E"               "$P6N"              
 [52] "$P6R"               "$P6TYPE"            "$P6V"              
 [55] "$P7B"               "$P7E"               "$P7N"              
 [58] "$P7R"               "$P7TYPE"            "$P7V"              
 [61] "$P8B"               "$P8E"               "$P8N"              
 [64] "$P8R"               "$P8TYPE"            "$P8V"              
 [67] "$P9B"               "$P9E"               "$P9N"              
 [70] "$P9R"               "$P9TYPE"            "$P9V"              
 [73] "$P10B"              "$P10E"              "$P10N"             
 [76] "$P10R"              "$P10TYPE"           "$P10V"             
 [79] "$PAR"               "$PROJ"              "$SPILLOVER"        
 [82] "$TIMESTEP"          "$TOT"               "$VOL"              
 [85] "APPLY COMPENSATION" "CHARSET"            "CREATOR"           
 [88] "FCSversion"         "FILENAME"           "flowCore_$P2Rmax"  
 [91] "flowCore_$P2Rmin"   "flowCore_$P3Rmax"   "flowCore_$P3Rmin"  
 [94] "flowCore_$P1Rmax"   "flowCore_$P1Rmin"   "flowCore_$P4Rmax"  
 [97] "flowCore_$P4Rmin"   "flowCore_$P5Rmax"   "flowCore_$P5Rmin"  
[100] "flowCore_$P6Rmax"   "flowCore_$P6Rmin"   "flowCore_$P7Rmax"  
[103] "flowCore_$P7Rmin"   "flowCore_$P8Rmax"   "flowCore_$P8Rmin"  
[106] "flowCore_$P9Rmax"   "flowCore_$P9Rmin"   "flowCore_$P10Rmax" 
[109] "flowCore_$P10Rmin"  "FSC ASF"            "GROUPNAME"         
[112] "GUID"               "LASER1ASF"          "LASER1DELAY"       
[115] "LASER1NAME"         "LASER2ASF"          "LASER2DELAY"       
[118] "LASER2NAME"         "LASER3ASF"          "LASER3DELAY"       
[121] "LASER3NAME"         "LASER4ASF"          "LASER4DELAY"       
[124] "LASER4NAME"         "LASER5ASF"          "LASER5DELAY"       
[127] "LASER5NAME"         "P2DISPLAY"          "P3DISPLAY"         
[130] "P1DISPLAY"          "P4DISPLAY"          "P5DISPLAY"         
[133] "P6DISPLAY"          "P7DISPLAY"          "P8DISPLAY"         
[136] "P9DISPLAY"          "P10DISPLAY"         "THRESHOLD"         
[139] "transformation"     "TUBENAME"           "USERSETTINGNAME"   
[142] "WINDOW EXTENSION"   "ORIGINALGUID"       "$P11B"             
[145] "$P11E"              "$P11N"              "$P11R"             
[148] "$P11S"              "$P11TYPE"           "$P11V"             
[151] "flowCore_$P11Rmax"  "flowCore_$P11Rmin"  "P11DISPLAY"        
[154] "$P12B"              "$P12E"              "$P12N"             
[157] "$P12R"              "$P12S"              "$P12TYPE"          
[160] "$P12V"              "flowCore_$P12Rmax"  "flowCore_$P12Rmin" 
[163] "P12DISPLAY"         "$P13B"              "$P13E"             
[166] "$P13N"              "$P13R"              "$P13S"             
[169] "$P13TYPE"           "$P13V"              "flowCore_$P13Rmax" 
[172] "flowCore_$P13Rmin"  "P13DISPLAY"         "$P14B"             
[175] "$P14E"              "$P14N"              "$P14R"             
[178] "$P14S"              "$P14TYPE"           "$P14V"             
[181] "flowCore_$P14Rmax"  "flowCore_$P14Rmin"  "P14DISPLAY"        
[184] "$P15B"              "$P15E"              "$P15N"             
[187] "$P15R"              "$P15S"              "$P15TYPE"          
[190] "$P15V"              "flowCore_$P15Rmax"  "flowCore_$P15Rmin" 
[193] "P15DISPLAY"         "$P16B"              "$P16E"             
[196] "$P16N"              "$P16R"              "$P16S"             
[199] "$P16TYPE"           "$P16V"              "flowCore_$P16Rmax" 
[202] "flowCore_$P16Rmin"  "P16DISPLAY"         "$P17B"             
[205] "$P17E"              "$P17N"              "$P17R"             
[208] "$P17S"              "$P17TYPE"           "$P17V"             
[211] "flowCore_$P17Rmax"  "flowCore_$P17Rmin"  "P17DISPLAY"        
[214] "$P18B"              "$P18E"              "$P18N"             
[217] "$P18R"              "$P18S"              "$P18TYPE"          
[220] "$P18V"              "flowCore_$P18Rmax"  "flowCore_$P18Rmin" 
[223] "P18DISPLAY"         "$P19B"              "$P19E"             
[226] "$P19N"              "$P19R"              "$P19S"             
[229] "$P19TYPE"           "$P19V"              "flowCore_$P19Rmax" 
[232] "flowCore_$P19Rmin"  "P19DISPLAY"         "$P20B"             
[235] "$P20E"              "$P20N"              "$P20R"             
[238] "$P20S"              "$P20TYPE"           "$P20V"             
[241] "flowCore_$P20Rmax"  "flowCore_$P20Rmin"  "P20DISPLAY"        
[244] "$P21B"              "$P21E"              "$P21N"             
[247] "$P21R"              "$P21S"              "$P21TYPE"          
[250] "$P21V"              "flowCore_$P21Rmax"  "flowCore_$P21Rmin" 
[253] "P21DISPLAY"         "$P22B"              "$P22E"             
[256] "$P22N"              "$P22R"              "$P22S"             
[259] "$P22TYPE"           "$P22V"              "flowCore_$P22Rmax" 
[262] "flowCore_$P22Rmin"  "P22DISPLAY"         "$P23B"             
[265] "$P23E"              "$P23N"              "$P23R"             
[268] "$P23S"              "$P23TYPE"           "$P23V"             
[271] "flowCore_$P23Rmax"  "flowCore_$P23Rmin"  "P23DISPLAY"        
[274] "$P24B"              "$P24E"              "$P24N"             
[277] "$P24R"              "$P24S"              "$P24TYPE"          
[280] "$P24V"              "flowCore_$P24Rmax"  "flowCore_$P24Rmin" 
[283] "P24DISPLAY"         "$P25B"              "$P25E"             
[286] "$P25N"              "$P25R"              "$P25S"             
[289] "$P25TYPE"           "$P25V"              "flowCore_$P25Rmax" 
[292] "flowCore_$P25Rmin"  "P25DISPLAY"         "$P26B"             
[295] "$P26E"              "$P26N"              "$P26R"             
[298] "$P26S"              "$P26TYPE"           "$P26V"             
[301] "flowCore_$P26Rmax"  "flowCore_$P26Rmin"  "P26DISPLAY"        
[304] "$P27B"              "$P27E"              "$P27N"             
[307] "$P27R"              "$P27S"              "$P27TYPE"          
[310] "$P27V"              "flowCore_$P27Rmax"  "flowCore_$P27Rmin" 
[313] "P27DISPLAY"         "$P28B"              "$P28E"             
[316] "$P28N"              "$P28R"              "$P28S"             
[319] "$P28TYPE"           "$P28V"              "flowCore_$P28Rmax" 
[322] "flowCore_$P28Rmin"  "P28DISPLAY"         "$P29B"             
[325] "$P29E"              "$P29N"              "$P29R"             
[328] "$P29S"              "$P29TYPE"           "$P29V"             
[331] "flowCore_$P29Rmax"  "flowCore_$P29Rmin"  "P29DISPLAY"        
[334] "$P30B"              "$P30E"              "$P30N"             
[337] "$P30R"              "$P30S"              "$P30TYPE"          
[340] "$P30V"              "flowCore_$P30Rmax"  "flowCore_$P30Rmin" 
[343] "P30DISPLAY"         "$P31B"              "$P31E"             
[346] "$P31N"              "$P31R"              "$P31S"             
[349] "$P31TYPE"           "$P31V"              "flowCore_$P31Rmax" 
[352] "flowCore_$P31Rmin"  "P31DISPLAY"         "$P32B"             
[355] "$P32E"              "$P32N"              "$P32R"             
[358] "$P32S"              "$P32TYPE"           "$P32V"             
[361] "flowCore_$P32Rmax"  "flowCore_$P32Rmin"  "P32DISPLAY"        
[364] "$P33B"              "$P33E"              "$P33N"             
[367] "$P33R"              "$P33S"              "$P33TYPE"          
[370] "$P33V"              "flowCore_$P33Rmax"  "flowCore_$P33Rmin" 
[373] "P33DISPLAY"         "$P34B"              "$P34E"             
[376] "$P34N"              "$P34R"              "$P34S"             
[379] "$P34TYPE"           "$P34V"              "flowCore_$P34Rmax" 
[382] "flowCore_$P34Rmin"  "P34DISPLAY"         "$P35B"             
[385] "$P35E"              "$P35N"              "$P35R"             
[388] "$P35S"              "$P35TYPE"           "$P35V"             
[391] "flowCore_$P35Rmax"  "flowCore_$P35Rmin"  "P35DISPLAY"        
[394] "$P36B"              "$P36E"              "$P36N"             
[397] "$P36R"              "$P36S"              "$P36TYPE"          
[400] "$P36V"              "flowCore_$P36Rmax"  "flowCore_$P36Rmin" 
[403] "P36DISPLAY"         "$P37B"              "$P37E"             
[406] "$P37N"              "$P37R"              "$P37S"             
[409] "$P37TYPE"           "$P37V"              "flowCore_$P37Rmax" 
[412] "flowCore_$P37Rmin"  "P37DISPLAY"         "$P38B"             
[415] "$P38E"              "$P38N"              "$P38R"             
[418] "$P38S"              "$P38TYPE"           "$P38V"             
[421] "flowCore_$P38Rmax"  "flowCore_$P38Rmin"  "P38DISPLAY"        
[424] "$P39B"              "$P39E"              "$P39N"             
[427] "$P39R"              "$P39S"              "$P39TYPE"          
[430] "$P39V"              "flowCore_$P39Rmax"  "flowCore_$P39Rmin" 
[433] "P39DISPLAY"         "$P40B"              "$P40E"             
[436] "$P40N"              "$P40R"              "$P40S"             
[439] "$P40TYPE"           "$P40V"              "flowCore_$P40Rmax" 
[442] "flowCore_$P40Rmin"  "P40DISPLAY"        

One thing we can do now that the new keyword() entries have been generated, is to use the order() and “[]” functions to rearrange them. We can also overwrite the existing parameters() “data” slot with our updated parameters data.frame, and convert the unmixed data to a matrix for eventual incorporation into exprs()

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' select
#' @importFrom tibble rownames_to_column column_to_rownames
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

        # Final renaming in action for "PDisplay"

        DisplayRenamePattern <- paste0("^(\\$)?P", OldXNum, "(.*)$")
        ThirdRenamed <- sub(DisplayRenamePattern, 
        paste0("\\1P", NewXNum, "\\2"), ThirdRename)

        names(ForLoopDescription)[match(ThirdRename,
         names(ForLoopDescription))] <- ThirdRenamed
    }

    # Add the new parameter rows for the Unmixed Fluorophores

    IntermediateParameters <- IntermediateParameters |>
         dplyr::select(-OriginalRowNumber) |>
         tibble::column_to_rownames("NewRowNumber")

    UpdatedParameters <- UnmixedParameterUpdate(OldParameters=IntermediateParameters, NewExprs=data)
    TheAntigens <- panel |> pull(Antigen)
    UpdatedParameters$desc <- TheAntigens
    pd <- rbind(IntermediateParameters, UpdatedParameters)

    # Generate new keywords for the new fluorophore rows in parameters

    new_pid <- rownames(UpdatedParameters)
    new_kw <- ForLoopDescription

    for (i in new_pid){
        NoDollarCode <- sub("^\\$", "", i) # For "PDisplay keyword"
        new_kw[paste0(i,"B")] <- new_kw["$P1B"] # Bytes
        new_kw[paste0(i,"E")] <- "0,0"
        new_kw[paste0(i,"N")] <- pd[[i,1]] # Fluorophore Name
        new_kw[paste0(i,"R")] <- pd[[i,5]] # Range Default
        new_kw[paste0(i,"S")] <- pd[[i,2]] # Antigen Name
        new_kw[paste0(i,"TYPE")] <- "Unmixed_Fluorescence"
        new_kw[paste0(i,"V")] <- "0" # Voltage/Gain, unmixed default 0
        new_kw[paste0("flowCore_", i,"Rmax")] <- pd[[i,5]] # maxRange
        new_kw[paste0("flowCore_", i,"Rmin")] <- pd[[i,4]] # minRange
        new_kw[paste0(NoDollarCode,"DISPLAY")] <- "LOG"
    }

    # Order Keywords by default sequence

    new_kw <- new_kw[order(names(new_kw))]

    # Overwrite old parameters "data" with the new parameters 

    OriginalParameterSlot <- flowCore::parameters(ff)
    OriginalParameterSlot@data <- pd

    # Convert unmixed data.frame to unmixed matrix

    UnmixedMatrix <- as.matrix(data)

    return(UnmixedMatrix)
}

Matrix <- UnmixInternal(ff=ff, data=data, panel=panel)
class(Matrix)

[1] "matrix" "array" 

One last major thing, within keyword(), we have a “$SPILLOVER” keyword that contains a spillover matrix. While we are unlikely to need it, we should go ahead and convert it from the current entry which has detector columns to one that has fluorophore columns to ensure compatability.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' select
#' @importFrom tibble rownames_to_column column_to_rownames
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

        # Final renaming in action for "PDisplay"

        DisplayRenamePattern <- paste0("^(\\$)?P", OldXNum, "(.*)$")
        ThirdRenamed <- sub(DisplayRenamePattern, 
        paste0("\\1P", NewXNum, "\\2"), ThirdRename)

        names(ForLoopDescription)[match(ThirdRename,
         names(ForLoopDescription))] <- ThirdRenamed
    }

    # Add the new parameter rows for the Unmixed Fluorophores

    IntermediateParameters <- IntermediateParameters |>
         dplyr::select(-OriginalRowNumber) |>
         tibble::column_to_rownames("NewRowNumber")

    UpdatedParameters <- UnmixedParameterUpdate(OldParameters=IntermediateParameters, NewExprs=data)
    TheAntigens <- panel |> pull(Antigen)
    UpdatedParameters$desc <- TheAntigens
    pd <- rbind(IntermediateParameters, UpdatedParameters)

    # Generate new keywords for the new fluorophore rows in parameters

    new_pid <- rownames(UpdatedParameters)
    new_kw <- ForLoopDescription

    for (i in new_pid){
        NoDollarCode <- sub("^\\$", "", i) # For "PDisplay keyword"
        new_kw[paste0(i,"B")] <- new_kw["$P1B"] # Bytes
        new_kw[paste0(i,"E")] <- "0,0"
        new_kw[paste0(i,"N")] <- pd[[i,1]] # Fluorophore Name
        new_kw[paste0(i,"R")] <- pd[[i,5]] # Range Default
        new_kw[paste0(i,"S")] <- pd[[i,2]] # Antigen Name
        new_kw[paste0(i,"TYPE")] <- "Unmixed_Fluorescence"
        new_kw[paste0(i,"V")] <- "0" # Voltage/Gain, unmixed default 0
        new_kw[paste0("flowCore_", i,"Rmax")] <- pd[[i,5]] # maxRange
        new_kw[paste0("flowCore_", i,"Rmin")] <- pd[[i,4]] # minRange
        new_kw[paste0(NoDollarCode,"DISPLAY")] <- "LOG"
    }

    # Order Keywords by default sequence

    new_kw <- new_kw[order(names(new_kw))]

    # Overwrite old parameters "data" with the new parameters 

    OriginalParameterSlot <- flowCore::parameters(ff)
    OriginalParameterSlot@data <- pd

    # Convert unmixed data.frame to unmixed matrix

    UnmixedMatrix <- as.matrix(data)

    # Last keyword overrides
    new_kw$`CREATOR` <- "CytometryInR version 1.0.0"

    TheColumnNames <- colnames(data)
    TheSpilloverNames <- TheColumnNames[!grepl("Time|FSC|SSC", TheColumnNames)]
    MatrixSize <- length(TheSpilloverNames)
    NewMatrix <- matrix(0, nrow = MatrixSize, ncol = MatrixSize, byrow = TRUE)
    diag(NewMatrix) <- 1
    colnames(NewMatrix) <- TheSpilloverNames
    new_kw$`$SPILLOVER` <- NewMatrix

    return(new_kw$`$SPILLOVER`)
}

UnmixInternal(ff=ff, data=data, panel=panel)

      BUV395-A BUV496-A BUV563-A BUV615-A BUV661-A BUV737-A BUV805-A BV421-A
 [1,]        1        0        0        0        0        0        0       0
 [2,]        0        1        0        0        0        0        0       0
 [3,]        0        0        1        0        0        0        0       0
 [4,]        0        0        0        1        0        0        0       0
 [5,]        0        0        0        0        1        0        0       0
 [6,]        0        0        0        0        0        1        0       0
 [7,]        0        0        0        0        0        0        1       0
 [8,]        0        0        0        0        0        0        0       1
 [9,]        0        0        0        0        0        0        0       0
[10,]        0        0        0        0        0        0        0       0
[11,]        0        0        0        0        0        0        0       0
[12,]        0        0        0        0        0        0        0       0
[13,]        0        0        0        0        0        0        0       0
[14,]        0        0        0        0        0        0        0       0
[15,]        0        0        0        0        0        0        0       0
[16,]        0        0        0        0        0        0        0       0
[17,]        0        0        0        0        0        0        0       0
[18,]        0        0        0        0        0        0        0       0
[19,]        0        0        0        0        0        0        0       0
[20,]        0        0        0        0        0        0        0       0
[21,]        0        0        0        0        0        0        0       0
[22,]        0        0        0        0        0        0        0       0
[23,]        0        0        0        0        0        0        0       0
[24,]        0        0        0        0        0        0        0       0
[25,]        0        0        0        0        0        0        0       0
[26,]        0        0        0        0        0        0        0       0
[27,]        0        0        0        0        0        0        0       0
[28,]        0        0        0        0        0        0        0       0
[29,]        0        0        0        0        0        0        0       0
[30,]        0        0        0        0        0        0        0       0
      Pacific Blue-A BV480-A BV510-A BV605-A BV650-A BV711-A BV750-A BV786-A
 [1,]              0       0       0       0       0       0       0       0
 [2,]              0       0       0       0       0       0       0       0
 [3,]              0       0       0       0       0       0       0       0
 [4,]              0       0       0       0       0       0       0       0
 [5,]              0       0       0       0       0       0       0       0
 [6,]              0       0       0       0       0       0       0       0
 [7,]              0       0       0       0       0       0       0       0
 [8,]              0       0       0       0       0       0       0       0
 [9,]              1       0       0       0       0       0       0       0
[10,]              0       1       0       0       0       0       0       0
[11,]              0       0       1       0       0       0       0       0
[12,]              0       0       0       1       0       0       0       0
[13,]              0       0       0       0       1       0       0       0
[14,]              0       0       0       0       0       1       0       0
[15,]              0       0       0       0       0       0       1       0
[16,]              0       0       0       0       0       0       0       1
[17,]              0       0       0       0       0       0       0       0
[18,]              0       0       0       0       0       0       0       0
[19,]              0       0       0       0       0       0       0       0
[20,]              0       0       0       0       0       0       0       0
[21,]              0       0       0       0       0       0       0       0
[22,]              0       0       0       0       0       0       0       0
[23,]              0       0       0       0       0       0       0       0
[24,]              0       0       0       0       0       0       0       0
[25,]              0       0       0       0       0       0       0       0
[26,]              0       0       0       0       0       0       0       0
[27,]              0       0       0       0       0       0       0       0
[28,]              0       0       0       0       0       0       0       0
[29,]              0       0       0       0       0       0       0       0
[30,]              0       0       0       0       0       0       0       0
      FITC-A Spark Blue 550-A PerCP-Cy5.5-A PE-A PE-Dazzle 594-A PE-Cy5-A
 [1,]      0                0             0    0               0        0
 [2,]      0                0             0    0               0        0
 [3,]      0                0             0    0               0        0
 [4,]      0                0             0    0               0        0
 [5,]      0                0             0    0               0        0
 [6,]      0                0             0    0               0        0
 [7,]      0                0             0    0               0        0
 [8,]      0                0             0    0               0        0
 [9,]      0                0             0    0               0        0
[10,]      0                0             0    0               0        0
[11,]      0                0             0    0               0        0
[12,]      0                0             0    0               0        0
[13,]      0                0             0    0               0        0
[14,]      0                0             0    0               0        0
[15,]      0                0             0    0               0        0
[16,]      0                0             0    0               0        0
[17,]      1                0             0    0               0        0
[18,]      0                1             0    0               0        0
[19,]      0                0             1    0               0        0
[20,]      0                0             0    1               0        0
[21,]      0                0             0    0               1        0
[22,]      0                0             0    0               0        1
[23,]      0                0             0    0               0        0
[24,]      0                0             0    0               0        0
[25,]      0                0             0    0               0        0
[26,]      0                0             0    0               0        0
[27,]      0                0             0    0               0        0
[28,]      0                0             0    0               0        0
[29,]      0                0             0    0               0        0
[30,]      0                0             0    0               0        0
      PE-Vio 770-A APC-A Alexa Fluor 647-A APC-R700-A APC-Fire 750-A
 [1,]            0     0                 0          0              0
 [2,]            0     0                 0          0              0
 [3,]            0     0                 0          0              0
 [4,]            0     0                 0          0              0
 [5,]            0     0                 0          0              0
 [6,]            0     0                 0          0              0
 [7,]            0     0                 0          0              0
 [8,]            0     0                 0          0              0
 [9,]            0     0                 0          0              0
[10,]            0     0                 0          0              0
[11,]            0     0                 0          0              0
[12,]            0     0                 0          0              0
[13,]            0     0                 0          0              0
[14,]            0     0                 0          0              0
[15,]            0     0                 0          0              0
[16,]            0     0                 0          0              0
[17,]            0     0                 0          0              0
[18,]            0     0                 0          0              0
[19,]            0     0                 0          0              0
[20,]            0     0                 0          0              0
[21,]            0     0                 0          0              0
[22,]            0     0                 0          0              0
[23,]            1     0                 0          0              0
[24,]            0     1                 0          0              0
[25,]            0     0                 1          0              0
[26,]            0     0                 0          1              0
[27,]            0     0                 0          0              1
[28,]            0     0                 0          0              0
[29,]            0     0                 0          0              0
[30,]            0     0                 0          0              0
      Zombie NIR-A APC-Fire 810-A AF-A
 [1,]            0              0    0
 [2,]            0              0    0
 [3,]            0              0    0
 [4,]            0              0    0
 [5,]            0              0    0
 [6,]            0              0    0
 [7,]            0              0    0
 [8,]            0              0    0
 [9,]            0              0    0
[10,]            0              0    0
[11,]            0              0    0
[12,]            0              0    0
[13,]            0              0    0
[14,]            0              0    0
[15,]            0              0    0
[16,]            0              0    0
[17,]            0              0    0
[18,]            0              0    0
[19,]            0              0    0
[20,]            0              0    0
[21,]            0              0    0
[22,]            0              0    0
[23,]            0              0    0
[24,]            0              0    0
[25,]            0              0    0
[26,]            0              0    0
[27,]            0              0    0
[28,]            1              0    0
[29,]            0              1    0
[30,]            0              0    1

And with that, everything in the exprs(), parameters() and keyword() slots has been converted over to match the expected unmixed .fcs file format. All that we need to do is cobble them together into a new flowFrame, which we can do via new(). At that point, at long last, we can return the new unmixed flowFrame back to OldFashionedUnmix(), where any last customization steps (as well as decisions where to store the .fcs file to) can occur without having a nested function layer to deal with.

#' Internal function for OldFashinedUnmix, handles fixing the 
#' formatting going from raw to unmixed .fcs file
#' 
#' @param ff The original raw flowFrame (used for the initial metadata)
#' @param data The updated data.frame following unmixing
#' @param panel Metadata from the signature matrix, containing
#'  Fluorophore and Antigen
#' 
#' @importFrom flowCore parameters keyword
#' @importFrom dplyr filter mutate row_number relocate pull
#' select
#' @importFrom tibble rownames_to_column column_to_rownames
# 
UnmixInternal <- function(ff, data, panel){

    # Identify the retained columns

    TheOriginalColumns <- colnames(ff)
    TheNewColumns <- colnames(data)
    RetainedColumns <- intersect(TheOriginalColumns, TheNewColumns)

    # Retrieve original parameters data with "$P" rownames

    TheOriginalParameters <- flowCore::parameters(ff)@data

    # Identify "$P" rownames to eliminate or keep

    KeepThese <- TheOriginalParameters |>
         dplyr::filter(name %in% RetainedColumns)
    GetRidThese <- TheOriginalParameters |> 
        dplyr::filter(!name %in% RetainedColumns) |> rownames()

    # Identify existing keyword names

    TheOriginalDescription <- flowCore::keyword(ff)
    OriginalKeywordNames <- names(TheOriginalDescription)

    # Identification of keywords containing "$P"

    Escaped <- gsub("\\$", "\\\\$", GetRidThese)
    RegexFormatted <- paste0("^", Escaped, "($|[^0-9])")
    RegexCombinatorial <- paste(RegexFormatted, collapse = "|")

    IdentifiedElimination <- OriginalKeywordNames[
        grepl(RegexCombinatorial, OriginalKeywordNames)]
    IdentifiedRetention <- OriginalKeywordNames[
        !grepl(RegexCombinatorial, OriginalKeywordNames)]

    # Identification of keywords containing "flowCore_$P"

    SecondRegexPattern <- paste0("(?<!\\d)", Escaped, "(?!\\d)")
    SecondRegexCombinatorial <- paste(SecondRegexPattern, collapse = "|")

    SecondElimination <- IdentifiedRetention[
        grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]
    SecondRetention <- IdentifiedRetention[
        !grepl(SecondRegexCombinatorial, IdentifiedRetention, perl = TRUE)]

    # Identification of keywords containing "PDisplay"

    NoDollars <- sub("^\\$", "", GetRidThese)
    NoDollarsRegex <- paste0("(?<!\\d)\\$?", NoDollars, "(?!\\d)")
    NoDollarsCombined <- paste(NoDollarsRegex, collapse = "|")

    ThirdElimination <- SecondRetention[
        grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]
    ThirdRetention <- SecondRetention[
        !grepl(NoDollarsCombined, SecondRetention, perl = TRUE)]

    # Subset original description for keyword names we want to retain

    IntermediateDescription <- TheOriginalDescription[ThirdRetention]

    # Renumbering retained SSC, FSC, SSC-B columns in parameters

    IntermediateParameters <- KeepThese |>
         tibble::rownames_to_column("OriginalRowNumber") |>
       dplyr::mutate(NewRowNumber=paste0("$P", dplyr::row_number())) |>
       relocate(NewRowNumber, .before=1)

    # Renumbering the retained keywords with new "$P" row numbers

    NewRowNumbers <- IntermediateParameters |>
         dplyr::pull(NewRowNumber)
    OriginalRowNumbers <- IntermediateParameters |>
         dplyr::pull(OriginalRowNumber)

    ForLoopDescription <- IntermediateDescription

    # For-loop to renumber the retained keywords

    for (i in seq_along(NewRowNumbers)) {
    
        NewX <- NewRowNumbers[i]
        NewXNum <- sub("^\\$P", "", NewX)

        OldX <- OriginalRowNumbers[i]
        OldXNum <- sub("^\\$P", "", OldX)

        # Rename matching keywords with new row numbers

        InternalEscaped <- gsub("\\$", "\\\\$", OldX)
        InternalRegexFormatted <- paste0(
            "(?<!\\d)", InternalEscaped, "(?!\\d)")
        InternalRegexCombinatorial <- paste(
            InternalRegexFormatted, collapse = "|")

        InternalIdentifiedRename <- names(ForLoopDescription)[
            grepl(InternalRegexCombinatorial, 
            names(ForLoopDescription), perl = TRUE)]

        ## Actual renaming in action

        RenamePattern <- paste0("^(flowCore_)?(\\$)?P", OldXNum, "(.*)$")
        InternalRenamed <- sub(RenamePattern,
         paste0("\\1\\2P", NewXNum, "\\3"), InternalIdentifiedRename)

        names(ForLoopDescription)[match(InternalIdentifiedRename,
         names(ForLoopDescription))] <- InternalRenamed

        # Matching for rename the "PDisplay" keywords
        InternalNoDollars <- sub("^\\$", "", OldX)   # "P18"
        InternalNoDollarsRegex <- paste0(
            "(?<!\\d)\\$?", InternalNoDollars, "(?!\\d)")

        ThirdRename <- names(ForLoopDescription)[
        grepl(InternalNoDollarsRegex, names(ForLoopDescription),
         perl = TRUE)]

        # Final renaming in action for "PDisplay"

        DisplayRenamePattern <- paste0("^(\\$)?P", OldXNum, "(.*)$")
        ThirdRenamed <- sub(DisplayRenamePattern, 
        paste0("\\1P", NewXNum, "\\2"), ThirdRename)

        names(ForLoopDescription)[match(ThirdRename,
         names(ForLoopDescription))] <- ThirdRenamed
    }

    # Add the new parameter rows for the Unmixed Fluorophores

    IntermediateParameters <- IntermediateParameters |>
         dplyr::select(-OriginalRowNumber) |>
         tibble::column_to_rownames("NewRowNumber")

    UpdatedParameters <- UnmixedParameterUpdate(OldParameters=IntermediateParameters, NewExprs=data)
    TheAntigens <- panel |> pull(Antigen)
    UpdatedParameters$desc <- TheAntigens
    pd <- rbind(IntermediateParameters, UpdatedParameters)

    # Generate new keywords for the new fluorophore rows in parameters

    new_pid <- rownames(UpdatedParameters)
    new_kw <- ForLoopDescription

    for (i in new_pid){
        NoDollarCode <- sub("^\\$", "", i) # For "PDisplay keyword"
        new_kw[paste0(i,"B")] <- new_kw["$P1B"] # Bytes
        new_kw[paste0(i,"E")] <- "0,0"
        new_kw[paste0(i,"N")] <- pd[[i,1]] # Fluorophore Name
        new_kw[paste0(i,"R")] <- pd[[i,5]] # Range Default
        new_kw[paste0(i,"S")] <- pd[[i,2]] # Antigen Name
        new_kw[paste0(i,"TYPE")] <- "Unmixed_Fluorescence"
        new_kw[paste0(i,"V")] <- "0" # Voltage/Gain, unmixed default 0
        new_kw[paste0("flowCore_", i,"Rmax")] <- pd[[i,5]] # maxRange
        new_kw[paste0("flowCore_", i,"Rmin")] <- pd[[i,4]] # minRange
        new_kw[paste0(NoDollarCode,"DISPLAY")] <- "LOG"
    }

    # Order Keywords by default sequence

    new_kw <- new_kw[order(names(new_kw))]

    # Overwrite old parameters "data" with the new parameters 

    OriginalParameterSlot <- flowCore::parameters(ff)
    OriginalParameterSlot@data <- pd

    # Convert unmixed data.frame to unmixed matrix

    UnmixedMatrix <- as.matrix(data)

    # Last keyword overrides
    new_kw$`CREATOR` <- "CytometryInR version 1.0.0"

    TheColumnNames <- colnames(data)
    TheSpilloverNames <- TheColumnNames[!grepl("Time|FSC|SSC", TheColumnNames)]
    MatrixSize <- length(TheSpilloverNames)
    NewMatrix <- matrix(0, nrow = MatrixSize, ncol = MatrixSize, byrow = TRUE)
    diag(NewMatrix) <- 1
    colnames(NewMatrix) <- TheSpilloverNames
    new_kw$`$SPILLOVER` <- NewMatrix

    new_fcs <- new("flowFrame", exprs=UnmixedMatrix, parameters=OriginalParameterSlot,
                 description=new_kw)

    return(new_fcs)
}

UnmixInternal(ff=ff, data=data, panel=panel)

flowFrame object 'DTR_2023_ILT_01-INF052-Ctrl_Antibody.1235515.fcs'
with 10000 cells and 40 observables:
               name      desc     range  minRange  maxRange
$P1            Time        NA   1428432         0   1428431
$P2           SSC-W        NA   4194304         0   4194303
$P3           SSC-H        NA   4194304         0   4194303
$P4           SSC-A        NA   4194304         0   4194303
$P5           FSC-W        NA   4194304         0   4194303
...             ...       ...       ...       ...       ...
$P36     APC-R700-A    CD107a   4194304      -111   4192506
$P37 APC-Fire 750-A      CD27   4194304      -111   4192506
$P38   Zombie NIR-A Viability   4194304      -111   4192506
$P39 APC-Fire 810-A      CD38   4194304      -111   4192506
$P40           AF-A             4194304      -111   4192506
443 keywords are stored in the 'description' slot

And with that, everything in the exprs(), parameters() and keyword() slots has been converted over to match the expected unmixed .fcs file format. All that we need to do is cobble them together into a new flowFrame, which we can do via new(). At that point, at long last, we can return the new unmixed flowFrame back to OldFashionedUnmix(), where any last customization steps (as well as decisions where to store the .fcs file to) can occur without having a nested function layer to deal with.

Take Away

In this bonus-walkthrough, we took our unmixed fluorophore outputs, and figured out how to modify the original raw .fcs file to remove the no longer needed elements related to the detector columns, reformat those elements that were retained, and then add in new elements related to each of the new fluorophores. This resulted in the creation of a new .fcs file that matched the typical formatting of an unmixed .fcs file.

To return to the main unmixing walk-through, click here