---
title: "Module 4 Theory — Date & Text Handling"
format:
  html:
    css: "../dark-theme.css"
    include-in-header: |
      <script>
        // Immediate theme application to prevent flash
        (function() {
          const savedTheme = localStorage.getItem('theme') || 'light';
          document.documentElement.setAttribute('data-bs-theme', savedTheme);
        })();
      </script>
    include-after-body: |
      <button class="theme-toggle" onclick="toggleTheme()" title="Toggle theme">🌙</button>
      <script>
        function toggleTheme() {
          const html = document.documentElement;
          const currentTheme = html.getAttribute('data-bs-theme');
          const newTheme = currentTheme === 'dark' ? 'light' : 'dark';

          html.setAttribute('data-bs-theme', newTheme);
          localStorage.setItem('theme', newTheme);

          // Update button icon
          const button = document.querySelector('.theme-toggle');
          button.innerHTML = newTheme === 'dark' ? '☀️' : '🌙';
        }

        // Initialize theme on load
        document.addEventListener('DOMContentLoaded', function() {
          const savedTheme = localStorage.getItem('theme') || 'light';
          document.documentElement.setAttribute('data-bs-theme', savedTheme);
          const button = document.querySelector('.theme-toggle');
          if (button) {
            button.innerHTML = savedTheme === 'dark' ? '☀️' : '🌙';
          }
        });
      </script>
editor: visual
---
# 📅 Module 4 — Dates, Text & Factors

## 🎯 Learning Objectives

By the end of this module, you will:

-   **Master string manipulation** with stringr for clinical text processing (R4DS Ch. 14)
-   **Apply regular expressions** for pattern matching in clinical data (R4DS Ch. 15)
-   **Work with factors** for categorical clinical data management (R4DS Ch. 16)
-   **Convert dates** using lubridate functions (ymd, dmy, mdy) for clinical data
-   **Calculate study days** (e.g., AESTDY = AESTDTC - RFSTDTC + 1)
-   **Clean and standardize** adverse event terms and medical coding
-   **Handle date/time formats** and missing date scenarios in clinical contexts

---
title: "Module 4 Theory — Date & Text Handling"
format:
  html:
    css: "../dark-theme.css"
    include-after-body: |
      <script src="../theme-manager.js"></script>
      <script>
        // Immediate theme application
        (function() {
          const savedTheme = localStorage.getItem('theme') || 'light';
          document.documentElement.setAttribute('data-bs-theme', savedTheme);
        })();
      </script>
editor: visual
---
## 2. Study Day Calculations

Study day calculations are fundamental in clinical programming. The formula is typically:
**Study Day = Event Date - Reference Start Date + 1**

### Basic Study Day Calculation

```r
# Sample adverse events data
ae_data <- tibble(
  USUBJID = c("001-001", "001-001", "001-002", "001-002"),
  AEDECOD = c("HEADACHE", "NAUSEA", "FATIGUE", "DIZZINESS"),
  AESTDTC = c("2024-01-20", "2024-01-25", "2024-01-18", "2024-01-22"),
  RFSTDTC = c("2024-01-15", "2024-01-15", "2024-01-16", "2024-01-16")
)

# Calculate AESTDY
ae_data <- ae_data %>%
  mutate(
    AESTDT = ymd(AESTDTC),      # Convert to Date
    RFSTDT = ymd(RFSTDTC),      # Convert to Date
    AESTDY = as.numeric(AESTDT - RFSTDT) + 1  # Calculate study day
  )

print(ae_data)
```

### Handling Different Scenarios

```r
# More complex study day calculation with validation
ae_data <- ae_data %>%
  mutate(
    AESTDY = case_when(
      is.na(AESTDT) | is.na(RFSTDT) ~ NA_real_,  # Handle missing dates
      AESTDT < RFSTDT ~ as.numeric(AESTDT - RFSTDT),  # Negative study days (pre-treatment)
      TRUE ~ as.numeric(AESTDT - RFSTDT) + 1     # Positive study days (post-treatment)
    )
  )
```

---
title: "Module 4 Theory — Date & Text Handling"
format:
  html:
    css: "../dark-theme.css"
    include-after-body: |
      <script src="../theme-manager.js"></script>
      <script>
        // Immediate theme application
        (function() {
          const savedTheme = localStorage.getItem('theme') || 'light';
          document.documentElement.setAttribute('data-bs-theme', savedTheme);
        })();
      </script>
editor: visual
---
## 🏷️ 5. Working with Factors (R4DS Ch. 16)

Factors are R's way of handling categorical data. In clinical programming, factors are essential for:

- **Treatment groups**: Controlling order in analyses and plots
- **Severity levels**: Ensuring proper ordering (Mild < Moderate < Severe)
- **Visit schedules**: Maintaining chronological order of visits
- **Categorical endpoints**: Managing ordered categories for efficacy analyses

### Creating and Managing Factors

```r
library(forcats)  # Part of tidyverse for factor manipulation

# Treatment groups with natural ordering
treatment <- c("Placebo", "Low Dose", "High Dose", "Placebo", "High Dose")
treatment_factor <- factor(treatment,
                          levels = c("Placebo", "Low Dose", "High Dose"))

# Check the levels
levels(treatment_factor)
# [1] "Placebo"   "Low Dose"  "High Dose"

# Adverse event severity with ordered levels
severity <- c("Mild", "Severe", "Moderate", "Mild", "Severe")
severity_factor <- factor(severity,
                         levels = c("Mild", "Moderate", "Severe"),
                         ordered = TRUE)

# Now we can make meaningful comparisons
severity_factor[1] < severity_factor[3]  # TRUE (Mild < Moderate)
```

### Clinical Factor Applications

```r
# Example: Visit factors for proper chronological ordering
visits <- c("Screening", "Baseline", "Week 4", "Week 8", "Week 12", "End of Study")
visit_factor <- factor(visits, levels = visits)  # Preserves order

# Example: Dose response categories
dose_response <- c("No Response", "Partial Response", "Complete Response")
response_factor <- factor(dose_response,
                         levels = dose_response,
                         ordered = TRUE)

# Using factors in clinical data processing
clinical_df <- tibble(
  USUBJID = c("001-001", "001-002", "001-003"),
  TRT01A = factor(c("Placebo", "Active", "Active"),
                  levels = c("Placebo", "Active")),
  AESEV = factor(c("MILD", "MODERATE", "SEVERE"),
                 levels = c("MILD", "MODERATE", "SEVERE"),
                 ordered = TRUE)
)

# Factors maintain proper ordering in summaries
clinical_df %>% count(TRT01A)  # Placebo appears first
clinical_df %>% count(AESEV)   # Maintains severity order
```

------------------------------------------------------------------------

## 🚧 6. Combining Date and Text Operations

### Complete AESTDY Derivation Example

```r
# Comprehensive example: derive AESTDY and clean AE terms
ae_complete <- tibble(
  USUBJID = c("001-001", "001-001", "001-002", "001-003"),
  AEDECOD = c("  Mild HEADACHE  ", "NAUSEA (moderate)", "severe FATIGUE", "Dizziness"),
  AESTDTC = c("2024-01-20", "2024-01-25", "2024-01-18", "2024-01-22"),
  RFSTDTC = c("2024-01-15", "2024-01-15", "2024-01-16", "2024-01-16")
) %>%
  mutate(
    # Date conversions and study day calculation
    AESTDT = ymd(AESTDTC),
    RFSTDT = ymd(RFSTDTC),
    AESTDY = as.numeric(AESTDT - RFSTDT) + 1,

    # Clean adverse event terms
    AEDECOD_CLEAN = AEDECOD %>%
      str_trim() %>%
      str_to_upper() %>%
      str_replace_all("\\([^)]*\\)", "") %>%  # Remove parentheses and contents
      str_replace_all("\\s+", " ") %>%        # Replace multiple spaces
      str_trim(),                             # Trim again after cleaning

    # Extract base term (remove severity qualifiers)
    AETERM_BASE = AEDECOD_CLEAN %>%
      str_replace_all("^(MILD|MODERATE|SEVERE)\\s+", ""),

    # Create study day categories
    AESTDY_CAT = case_when(
      AESTDY <= 0 ~ "Pre-treatment",
      AESTDY <= 7 ~ "Week 1",
      AESTDY <= 14 ~ "Week 2",
      TRUE ~ "After Week 2"
    )
  )

print(ae_complete)
```

------------------------------------------------------------------------

## 📅 7. Advanced Date Handling

### Working with Time Components

```r
# Handling date-time data
datetime_data <- tibble(
  USUBJID = c("001-001", "001-002"),
  AESTDTC = c("2024-01-20T08:30:00", "2024-01-21T14:15:30"),
  RFSTDTC = c("2024-01-15T09:00:00", "2024-01-16T10:30:00")
) %>%
  mutate(
    # Parse date-time
    AESTDT = ymd_hms(AESTDTC),
    RFSTDT = ymd_hms(RFSTDTC),

    # Extract components
    AE_DATE = date(AESTDT),
    AE_TIME = format(AESTDT, "%H:%M:%S"),
    AE_HOUR = hour(AESTDT),

    # Calculate study day from dates only (ignoring time)
    AESTDY = as.numeric(date(AESTDT) - date(RFSTDT)) + 1,

    # Time-based categories
    TIME_PERIOD = case_when(
      AE_HOUR < 12 ~ "Morning",
      AE_HOUR < 18 ~ "Afternoon",
      TRUE ~ "Evening"
    )
  )
```

```

---
title: "Module 4 Theory — Date & Text Handling"
format:
  html:
    css: "../dark-theme.css"
    include-after-body: |
      <script src="../theme-manager.js"></script>
      <script>
        // Immediate theme application
        (function() {
          const savedTheme = localStorage.getItem('theme') || 'light';
          document.documentElement.setAttribute('data-bs-theme', savedTheme);
        })();
      </script>
editor: visual
---
## ⚠️ 9. Common Pitfalls and Best Practices

### Date Handling Best Practices

```r
# ✅ Good practices
ae_data <- ae_data %>%
  mutate(
    # Always handle missing dates explicitly
    AESTDY = case_when(
      is.na(ymd(AESTDTC)) | is.na(ymd(RFSTDTC)) ~ NA_real_,
      TRUE ~ as.numeric(ymd(AESTDTC) - ymd(RFSTDTC)) + 1
    ),

    # Check for impossible dates
    DATE_FLAG = case_when(
      ymd(AESTDTC) < ymd("1900-01-01") ~ "INVALID",
      ymd(AESTDTC) > today() ~ "FUTURE",
      TRUE ~ "VALID"
    )
  )

# ❌ Avoid these issues
# Don't assume all dates parse correctly
# Don't forget the +1 in study day calculations for most sponsors
# Don't ignore time zones if working with datetime data
```

### String Handling Best Practices

```r
# ✅ Good practices for text cleaning
clean_ae_terms <- function(terms) {
  terms %>%
    str_trim() %>%                           # Remove whitespace
    str_to_upper() %>%                       # Standardize case
    str_replace_all("\\s+", " ") %>%         # Normalize spacing
    str_replace_all("[^A-Z0-9 ]", "") %>%    # Remove special characters
    str_trim()                               # Trim again after cleaning
}
```

------------------------------------------------------------------------

## 📝 Module Summary

By completing this module, you should now be able to:

✅ **Convert dates** using lubridate functions (ymd, dmy, mdy) for various input formats
✅ **Calculate study days** accurately with proper handling of missing dates
✅ **Manipulate text** using stringr functions for data cleaning and standardization
✅ **Use regular expressions** for advanced pattern matching in clinical data
✅ **Work with factors** for categorical variables like treatment groups and severity levels
✅ **Derive AESTDY** and clean adverse event terms in realistic clinical scenarios
✅ **Handle edge cases** like missing dates, invalid dates, and messy text data

### 🔗 R4DS Connections:
This module covers essential concepts for clinical programming:

- **Strings**: Essential string manipulation with stringr
- **Regular expressions**: Pattern matching for data validation
- **Factors**: Categorical data handling with forcats

### 🚀 Next Steps:
- Practice with the demo exercises
- Try deriving study days with your own clinical data
- Apply string patterns and factor management to real clinical datasets
- Prepare for Module 5: Functions & Macro Translation

------------------------------------------------------------------------

## 💡 Key Takeaways

1. **lubridate makes date parsing intuitive** - ymd(), dmy(), mdy() handle most formats automatically
2. **Study day = Event Date - Reference Date + 1** is the standard clinical calculation
3. **stringr provides consistent string manipulation** with clear, readable function names
4. **Always handle missing data explicitly** when working with dates and text
5. **GitHub Copilot excels** at suggesting date/text transformations for clinical programming
6. **Combine date and text operations** for comprehensive data cleaning pipelines

Ready to learn about writing functions? Let's move to Module 5!
