SciDataReportR 20.24.0
Clustering API is standardized around fitted pipelines and one projector
Breaking unreleased clustering-output cleanup. Finalized fits and projections now return
DataWithClusters(formerlydf_with_clusters) andClusterVariableName(formerlyClusterName). The mistypedModelInfo_Mclust, publiccomplete_rows,CandidateAudit, and injected.scidr_rowidcolumns have been removed.ModelInfo_MClustis the canonical Mclust-specific layer;ModelInforemains the generic compatibility alias.SOM + Mclust now reports
MinProfileNodeN/MaxProfileNodeNalongside their explicit proportions, andBLRTStatistic/BLRTPValue. The candidate plot includes an unselected BLRT p-value panel with a neutral 0.05 reference line.Clustering constructors now use
CreateClusterModel_<reduction>_<method>(), for exampleCreateClusterModel_PCA_MClust()andCreateClusterModel_SOM_HDBSCAN(). Fitted objects use matchingPipeline_<reduction>_<method>classes.ProjectCluster(object, new_df)is the single public projection interface. It dispatches from the fitted pipeline class and retains each method’s frozen preprocessing, reduction, support checks, and projection diagnostics.CreateSOMClusterModel(),ProjectSOMCluster(),Pipeline_SOMClust(), andProject_SOMClust()remain for established SOM compatibility. The first two emit lifecycle deprecation warnings and delegate to the canonical API.
The SciData palette is now the default categorical color scheme
Every plot the package produces now draws categorical color from
SciDataPalette(). Previously each function chose its own: some fell back to ggplot2’s default hue palette, others hardcoded a tableau-style vector, anhcl.colors()name, or apaletteerstring. Clusters, groups, cohorts, and category levels now look like one system across the whole package.Only categorical color changed. Continuous and diverging scales (correlation and effect-size heatmaps, the p-value scales) are untouched, as are the meaning-carrying scales: PASS/WARNING/FAIL status, the signed significance ladders, RCI classification, volcano significance tiers, the grey used for missing data, and the grey reserved for the density-based
Noisecluster.Fixes a latent failure.
CreatePCATable()andCreateZScorePlot()shared a 20-color vector that went straight intoscale_color_manual(), so both failed withInsufficient values in manual scaleonce a plot had more than 20 variable categories. Package plots now extend the palette instead of erroring, however many categories a variable has.Labels drawn inside filled shapes now pick black or white by background luminance.
PlotCategoricalDistributions()draws its labels inside the bars, and the palette’s darker colors would have made them unreadable.Palette arguments keep their existing signatures.
PlotClusterBoxplot(Palette),PlotSplitViolin(color_palette),PlotSpiderChart(Palette)andPlot2GroupStats(palette)all still accept exactly what they accepted before; only the default changed.PlotSpiderChart()andPlot2GroupStats()now default toNULL, meaning “use the package palette” - passing"Dark 3"or"pals::alphabet"explicitly works as it always did.SciDataPalette()itself is unchanged, including erroring when asked for more than its 34 colors.
Cluster occupancy figures removed from the clustering pipelines
The
occupancybar chart is no longer attached toModelInfo$plotsbyCreateClusterModel_MClust(),CreateClusterModel_KMeans(),CreateClusterModel_PCA_MClust(),CreateClusterModel_PCA_KMeans(),CreateClusterModel_HDBSCAN(),CreateClusterModel_Gower_PAM(),CreateClusterModel_LatentClass(),CreateClusterModel_MCA_MClust(), orCreateSOMClusterModel(). Cluster sizes are already reported in the fit tables and inProbFit$individual, so the figure added a panel without adding information.PlotClusterOccupancy()is unchanged and still exported, so the plot can still be drawn directly from a model’sDataWithClusters.The training-versus-projected occupancy comparison in the
Project*functions is unaffected: it compares two distributions rather than describing one, which is a projection-quality check rather than a restatement of the cluster sizes.
Symmetric p-value matrices are corrected once per pair
ApplyFDRCorrection()treated a symmetric matrix - the shape produced whenever a variable set is correlated against itself, asPlotCorrelationsHeatmap()does whenoutcome_varsis leftNULL- as a family ofn * (n - 1)tests, when onlyn * (n - 1) / 2were run. Newsymmetricargument, defaulting to"auto", detects the case and corrects each pair once, mirroring the adjusted values back so the matrix stays symmetric. Passsymmetric = FALSEfor the old behavior orsymmetric = TRUEto require it.Benjamini-Hochberg is invariant to exact duplication, so
method = "fdr"results - the default, and what every plotting function uses - are unchanged."bonferroni","holm"and"hochberg"were coming out exactly twice as large as they should have been, and are now correct.The diagonal of a symmetric matrix holds self-comparisons that were never tested. It is now excluded from the family and returned as
NA; setinclude_diagonal = TRUEto keep the old behavior. This was the damaging case: where the diagonal carried real values, a self-correlation p-value of 0 entered the family as the most significant test in it and pulled every off-diagonal result down, making findings look stronger than they were.
Misspelled variables are now an error instead of a silent result change
The matrix and heatmap functions used to handle an unknown variable name five different ways.
PlotCorrelationsHeatmap(),PlotAnovaRelationshipsMatrix(),PlotChiSqCovar(),PlotMiningMatrix(),PlotNumInteractionEffectsMatrix()andPlotCatInteractionEffectsMatrix()dropped the name silently, returning a smaller results matrix with no warning that anything had gone missing;PlotPhiHeatmap()andPlotPointCorrelationsHeatmap()failed with base R’s"undefined columns selected", which never named the offending variable; andPlotDirectionalHeatmaps()andPlotSpiderChart()each raised their own differently worded error. All of them now stop with one message that names both the variable and the argument it was supplied to, for exampleVariables not found indata: NOPE (supplied topredictor_vars).A misspelled covariate was the more serious case.
PlotCorrelationsHeatmap(),PlotChiSqCovar(),PlotAnovaRelationshipsMatrix(),PlotMiningMatrix()andPlotNumInteractionEffectsMatrix()silently ignored a covariate that did not match a column, so the function computed and reported unadjusted statistics while the caller believed they were adjusted. Unknown covariates now error.Validation happens against the data frame as supplied, before ordinal handling renames anything, so variables legitimately removed by
TreatOrdinalAsare unaffected. Auto-detected variable sets (variables = NULL) still work as before, andPlotCorrelationsHeatmap()still reports covariates dropped by ordinal handling throughCovariatesMissing.New internal helpers
ScidrValidateVariables()andScidrValidateVariable()provide the single message format, replacing theunique(intersect(as.character(vars), names(Data)))idiom that caused the silent dropping.
Significance stars agree across the package
Six separate star implementations disagreed on whether the cut points were inclusive: a p-value of exactly 0.001 earned
***inMultivariableRegressionTable()andPlotInteractionEffectsMatrix()but only**inPlotSplitViolin(), while every plot caption stated the strict<form regardless. All of them now route through oneScidrPValueStars()using inclusive upper bounds (p <= 0.05is*), which matchesrstatix::add_significance()— already used byPlotPhiHeatmap(),PlotPointCorrelationsHeatmap()andPlotAnovaRelationshipsMatrix()— and is verified against it at every boundary.Captions and legend labels now state
<=to match the thresholds actually applied. This affectsgeom_starcaption(),MakePairwiseHeatmap()captions,PlotMiningMatrix()legend labels, and thePlot2GroupStats()shape legend (whosecut(right = TRUE)breaks were always inclusive despite reading<).
Other fixes
PlotDirectionalHeatmaps()had the deprecatedyVarsin formal position 3, soPlotDirectionalHeatmaps(df, vars, TRUE)boundTRUEtoyVarsrather thanRelabel. All deprecated formals are now trailing, as elsewhere in the package.PlotDirectionalHeatmaps()’sOrdinalargument was documented as “reserved for future use” and as not affecting the computed tiles, but it is passed toPlotPointCorrelationsHeatmap()and does change how ordinal variables are treated in the binary~continuous block. The documentation now describes what it actually does.
SciDataReportR 20.23.0
Clustering pipelines no longer return a flat
plotslist. Figures are stored beside the object they describe, following the layoutCreateSOMClusterModel()already used:fit_plotreviews candidates,ModelInfo$plotsdescribes the selected solution,ModelInfo$FitDiagnostics$plotsdescribes how individual training cases sit inside it,ProbFit$plotsdescribes membership confidence,Stability$plotsdescribes bootstrap reproducibility, andProjectionFit$plotsdescribes projected cases against the frozen training reference. The previousplotslist mixed aliases, duplicated entries under two names, and stored interactive widgets alongsideggplotobjects; it has been removed.Every method now carries figures appropriate to that method rather than a shared lowest common denominator. K-means and Gower/PAM gain per-participant silhouette profiles, elbow and average-silhouette curves; Mclust gains BIC, ICL, entropy, and classification-uncertainty maps; HDBSCAN gains density-grid review and per-cluster persistence; latent class analysis gains item response profiles; and the PCA/MCA pipelines lead with scree and loadings. All methods gain a frozen two-dimensional review map, cluster centre profiles, and a distance histogram/boxplot/ECDF triad against a frozen high-distance cutoff.
Project*Cluster()results gainProjectionFit, which triages every projected case against the frozen training reference intoGood fit,Uncertain membership,Poor fit to training structure, orPotential novel phenotype, and addsProjection_Fit_ClasstoDataWithClusters. This extends the triageProjectSOMCluster()already performed to all clustering methods.CreateSOMClusterModel()andProjectSOMCluster()keep their existing figures unchanged. The SOM model gainsModelInfo_SOM$plots$occupancyandStability$plots; the projection exposes its diagnostics asProjectionFit(still also available asSOMProj) and gainssom_grid_map.CreateSOMClusterModel(method = "finalize", stability_resamples > 0)now refits through the exploratory path by a direct recursive call rather than evaluating a reconstructed call in the caller’s frame, which failed when arguments were supplied as local variables.Bootstrap stability resamples now preserve observed factor levels, and the MCA pipeline freezes its reduction on observed categories only. Previously every MCA stability replicate could fail when a resample dropped a rare category, and a declared-but-unobserved factor level made projection fail outright.
ProjectCluster()now warns and treats categories absent from the training model as incomplete cases instead of erroring.New exported figure helpers:
PlotClusterOccupancy(),PlotClusterMap(),PlotClusterSilhouette(),PlotClusterCentreHeatmap(),PlotClusterCentreProfile(),PlotClusterComposition(), andPlotClusterDiagnostic().PlotProjectionDiagnostics()has been renamed toPlotClusterDiagnostic().
SciDataReportR 20.15.0
-
MakeComparisonTable()now reports absolute Cohen’s d for two-group parametric comparisons, including covariate-adjusted d from estimated marginal means, and Cohen’s f for multi-group omnibus comparisons. Effect-size captions identify the scales present, provide qualified magnitude guides, and warn when d and f should not be compared numerically.
SciDataReportR 20.12.0
-
PlotVolcanoEffects()now reports more detail in itsResultsTableand point tooltips. Continuous outcomes gainR(zero-order Pearson correlation) andAdjustedR(covariate-adjusted partial correlation). Two-group categorical outcomes gainGroup1Level,Group2Level,Group1Mean, andGroup2Mean(raw predictor means within each group). These values also appear in theTooltipcolumn so they show up when the plot is passed toplotly::ggplotly(tooltip = "text"). - New
FreezeTableHeader()wraps agtsummarytable (or data frame) so its header row stays frozen while scrolling long tables in HTML Quarto/R Markdown output. -
MakeComparisonTable()now uses safe internal names for its pairwise columns. Previously contrast labels such as"3 - 1"produced column names likepw_X3...1, whose trailing...1collided with tidyverse name-repair and broke downstream tibble round-trips (for examplegtsummary::as_kable_extra(), and thereforeFreezeTableHeader()). Displayed contrast headers are unchanged. -
MakeUnivariateRegressionTable()now extracts regression results directly from model coefficient tables and formats the final display withgt, avoiding the slow per-modelgtsummary::tbl_regression()path. Fitted model objects are now skipped by default; setReturnModels = TRUEto include them inModelSummaries. -
ReadSciData()now usesdata.table::fread()for ordinary delimited files when available, which substantially speeds up large.csv,.tsv, and.txtimports. Setfast_delimited = FALSEto force the previousreadrpath.
SciDataReportR 20.5.0
-
UnivariateRegressionTable()was renamed toMakeUnivariateRegressionTable()andplotForestFromTable()was renamed toPlotForestFromTable()to match the package’s naming conventions. The old names remain available as backwards-compatible synonyms (soft-deprecated). -
MakeUnivariateRegressionTable()now returns aResultselement: a tidy dataframe of estimates, confidence intervals, and p-values, one row per term. -
PlotForestFromTable()now plots from theResultsdataframe, and also accepts a (filtered)Resultsdataframe directly. Objects created by older package versions still plot via a fallback extraction.
SciDataReportR 17.0.0
Dependency stability
- Internalized the minimal half-violin geom used by
PlotSplitViolin()so the split-violin workflow no longer depends on the archivedgghalvespackage.
SciDataReportR 16.25.0
Documentation and infrastructure positioning
- Repositioned the package as scientific workflow infrastructure for reproducible life science data reporting.
- Expanded the README with workflow families, a visualization gallery, implemented methods, function dependency chains, and future heatmap/volcano plot roadmap items.
- Reorganized pkgdown navigation and reference topics around workflows, visualization functions, projection chains, and reusable infrastructure.
- Clarified the downstream relationship between
PlotCorrelationsHeatmap(),add_r_and_stars(), andgeom_starcaption(). - Added a Quarto getting-started article based on the R/Medicine workflow narrative.
Workflow-oriented public names
- Added canonical workflow-oriented names for reusable objects, models, projections, plots, and metadata workflows.
- Added
CreatePCAObject(),CreateMCAObject(),PlotZScore(),PlotPathway_KT(),MakeDataDictionary(),ProjectZScore(),ProjectSOMCluster(),CreateZScoreObject(),CreateMScoreObject(),CreateNormativeTScoreModel(), andCreateSOMClusterModel(). - Preserved backward compatibility: the older public names remain exported as wrappers and do not emit lifecycle warnings.
- Documented
PlotPathway_KT()as a SciDataReportR visualization that is expected to move to a future metabolomics-focused package.