# Learning TMLE: Notes on Influence Functions and Doubly Robust Estimation

Editable English course notes. Formulae retain their TeX source; Julia and R code is read from executable files. See index.html for computed illustrations and experiments.

# Chapter 0: Getting to know the EIF through doubly robust estimation

In the SUPPORT/RHC cohort, 1918 of 5735 patients died within 30 days. Our question goes beyond the overall mortality proportion: **in the same target population, how much does the average 30-day mortality risk differ between receiving right heart catheterisation (RHC) and not receiving RHC?**

Patients who received RHC may have had different baseline characteristics from those who did not. To make this comparison, we need to compare their risks using the same distribution of baseline characteristics. Doubly robust estimation offers a way to make this adjusted comparison.

Below, $W$ denotes baseline characteristics, $A$ indicates RHC (1 for RHC, 0 for no RHC), and $Y$ indicates death within 30 days (1 for death, 0 for survival). Interpreting the adjusted risk difference as an average treatment effect (ATE) additionally requires the identification conditions of consistency, no unmeasured confounding given $W$, and a positive chance of either treatment at each relevant baseline profile. The course demonstrates calculations using real records; the case-study page brings together the [study design, adjustment variables and scope of causal interpretation](case-study.html).

## 1. Use two regressions to predict mortality risk and treatment probability

The two regressions have different response variables. Both can be fitted using familiar logistic regression.

**The outcome regression $Q$ predicts mortality risk.** Fit a regression of $Y$ on $A$ and $W$. After fitting, retain each person’s baseline characteristics and set $A$ in the prediction input first to 1 and then to 0, giving $\widehat Q(1,W)$ and $\widehat Q(0,W)$. Subtract the two predictions for each person, then average over all 5735 patients. This gives the regression standardisation estimate, also called g-computation.

**The treatment model $g$ predicts the probability of receiving RHC.** Fit a regression of $A$ on $W$ to obtain the propensity score $\widehat g(W)$. Inverse probability weighting (IPW) uses this to weight observed outcomes: $1/\widehat g(W)$ in the RHC group and $1/[1-\widehat g(W)]$ in the no-RHC group. Divide each group’s weighted outcome total by the total number of patients, then subtract to estimate a risk difference for the same population.

<div class="dr-intro-models"></div>

The two routes depend on different models: g-computation relies on outcome risk predictions, whereas IPW relies on treatment probability predictions. Since we can learn both from these data, we can try combining them.

## 2. Add weighted residuals to predictions to obtain a doubly robust estimate

Keep the risk difference estimate from the fitted $Q$, then examine the difference between each observed outcome and its prediction under the observed treatment. For patient $i$, this residual is $Y_i-\widehat Q(A_i,W_i)$. Augmented inverse probability weighting (AIPW) uses $g$ to weight residuals: divide an RHC recipient’s residual by their treatment probability, and a non-recipient’s residual by their probability of no treatment, with a negative sign in the risk difference. Finally, add the average of these contributions to the initial estimate.

$$
\widehat\psi_{\mathrm{AIPW}}
=\underbrace{\widehat\psi_{\mathrm{gcomp}}}_{\text{average predicted risk difference}}
+\underbrace{\frac1n\sum_{i=1}^n C_i}_{\text{average weighted residual}}.
$$

> **Notation recap:** $\widehat\psi$ is the sample estimate of the risk difference; $n=5735$ is the total number of patients and $i$ indexes patient rows. $C_i$ is the signed weighted residual just defined: $(Y_i-\widehat Q(1,W_i))/\widehat g(W_i)$ in the RHC group and $-(Y_i-\widehat Q(0,W_i))/[1-\widehat g(W_i)]$ in the no-RHC group. A hat marks a fitted or sample-calculated quantity; multiplying the risk difference by 100 expresses it in percentage points.

The key property of this combination is **double robustness**: under identification, overlap and the usual regularity conditions for estimation, AIPW consistently estimates the target risk difference if either $Q$ or $g$ is consistently estimated. Here, “consistent” means approaching the true target as the sample size increases.

- **If $Q$ is correct:** conditional on baseline characteristics and treatment, the residual has mean zero. The population average of the weighted residual is therefore also zero, preserving the correct risk predictions.
- **If $g$ is correct:** weighting makes the residuals under both treatments refer to the common baseline population. Their average can make up for the bias in $Q$’s predicted risks.

Either model can therefore provide a route to consistent estimation; if both are incorrect, the result may still be biased. “Mean zero” and “making up for bias” above describe population relationships. The correction in a finite sample will generally still fluctuate.

## 3. The EIF tells us how to construct this correction

Why use precisely these weighted residuals? To answer, we need the influence function (IF). **For a fixed estimation target and reference distribution, an IF maps one observation to a number.** Its value describes, to first order, how the target changes when the distribution gives slightly more weight to records of that kind. For this risk difference, the input is one patient’s $(W,A,Y)$; the output accounts for their outcome residual, treatment probability and predicted risk difference under the two treatments.

The efficient influence function (EIF) is the IF that determines the efficiency bound in a given statistical model. It lets us use the same function for two purposes: **constructing a correction to an estimate and studying its uncertainty.** Under the relevant conditions, its variance determines the attainable asymptotic efficiency bound for estimation error. The following chapters build this rule step by step, starting with the mean.

For the current risk difference, calculate each person’s EIF value using the fitted $Q$, $g$ and initial risk difference, then average over the observed records. The result is exactly the average weighted residual above. This uses the **empirical distribution**: each observed patient record receives weight $1/n$, and we average with these weights. “Empirical” means based on the current sample; in the RHC cohort, $n=5735$, so the death records have total weight $1918/5735$. An EIF has mean zero under the distribution to which it belongs. Here we evaluate the fitted distribution’s EIF on the original patient records, so its average can be non-zero and serve as a first-order correction.

<div class="dr-intro-correction"></div>

## 4. AIPW and TMLE apply the same rule in different places

**AIPW corrects the final estimate.** Keep the initial $Q$ and $g$, and add the correction just calculated to the g-computation result.

**Targeted maximum likelihood estimation (TMLE) updates $Q$ and then standardises again.** The update changes the estimated conditional mean function. Keep the initial regression coefficients and fitted $g$ fixed, then estimate a new coefficient that adjusts predictions in the direction specified by the EIF. Choose this coefficient to bring the average updated weighted residual close to zero, then calculate the risk difference from the new function’s predictions under both treatments. For a binary outcome, the update takes place on the logit scale, keeping predictions between 0 and 1. Standard TMLE also has double robustness under the relevant conditions.

<div class="dr-intro-routes"></div>

AIPW corrects the final estimate; TMLE adjusts the risk predictions used to calculate it. The similar results in the figure reflect these particular fits and data; assessing accuracy and confidence intervals also requires examining model fitting and the conditions for inference.

## 5. Complete TMLE using the same patient table

We have seen where the two corrections enter. Now follow TMLE’s six steps. First prepare the predictions and fit the update size. Next update risks under both treatments and average their differences. Finally, use the EIF to quantify uncertainty. Every step retains the same patients, with new columns highlighted in blue; all fitting and summaries use all 5735 patients.

<!-- tmle-step:0 -->
### ① Add three columns of initial risk predictions

We have already fitted the outcome regression. Now attach its outputs to the patient table: observed-treatment predictions will give residuals; predictions under both treatments will compare risks in the same baseline population. All three columns come from the same initial $Q$.

$
\widehat Q_{A,i}=\widehat Q(A_i,W_i),\qquad
\widehat Q_{1,i}=\widehat Q(1,W_i),\qquad
\widehat Q_{0,i}=\widehat Q(0,W_i).
$

> **Notation recap:** $i$ is the original patient row; $W_i$ is that person’s baseline and $A_i$ their observed treatment. Compare $\widehat Q_{A,i}$ with observed outcome $Y_i$; $\widehat Q_{1,i}$ and $\widehat Q_{0,i}$ predict mortality risk with RHC and no RHC as inputs respectively. Subscripts 0 and 1 denote treatment levels; hats denote fitted quantities, and all predictions are probabilities.

**Pass to the next step:** one observed-treatment prediction column and two prediction columns for standardisation.

<!-- tmle-step:1 -->
### ② Identify $H$ in AIPW’s weighted residuals

Step ① retained the initial risk predictions. Return to AIPW in §2 of this chapter: first calculate the standardised risk difference, then add the average patient-level weighted residual $C_i$. The earlier group-specific definitions of $C_i$ for RHC and no RHC combine into one line:

$
C_i=\underbrace{\left\{\frac{A_i}{\widehat g_i}
-\frac{1-A_i}{1-\widehat g_i}\right\}}_{H_{A,i}}
(Y_i-\widehat Q_{A,i}).
$

> **Notation recap:** $C_i$ is patient $i$’s correction contribution; $Y_i$ is observed mortality and $\widehat Q_{A,i}=\widehat Q(A_i,W_i)$ the initial prediction. The fitted RHC probability is $\widehat g_i=\widehat g(W_i)$. Since $A_i$ is 0 or 1, the braces automatically select the relevant group’s weight. Denote this coefficient multiplying the residual by $H_{A,i}$.

**$H_{A,i}$ is the signed inverse probability weight used in IPW above.** IPW multiplies it by the observed outcome; AIPW multiplies it by “observed outcome minus initial prediction”. For the later update of both treatment predictions, retain the coefficients for both inputs:

$
H_{1,i}=\frac1{\widehat g_i},\qquad
H_{0,i}=-\frac1{1-\widehat g_i},\qquad
H_{A,i}=A_iH_{1,i}+(1-A_i)H_{0,i}.
$

> **Notation recap:** $H_{1,i}$ and $H_{0,i}$ correspond to RHC and no-RHC inputs; $H_{A,i}$ selects one according to observed treatment. The weight $H$ is dimensionless; its minus sign comes from “RHC risk minus no-RHC risk”. In the TMLE update, this coefficient is called the clever covariate.

<div class="tmle-ipw-placeholder"></div>

**Now put the same $C_i$ back into the initial fitted EIF.** It also includes the deviation of the individual predicted risk difference from the overall average:

$
\widehat D_i^{\mathrm{init}}
=\underbrace{C_i}_{H_{A,i}(Y_i-\widehat Q_{A,i})}
+(\widehat Q_{1,i}-\widehat Q_{0,i})
-\widehat\psi_{\mathrm{gcomp}}.
$

> **Notation recap:** $\widehat D_i^{\mathrm{init}}$ is the initial fitted EIF’s value for record $i$; $\widehat Q_{1,i}-\widehat Q_{0,i}$ is this person’s initial predicted risk difference, and $\widehat\psi_{\mathrm{gcomp}}$ the average predicted risk difference across all 5735 patients. All $Q$, $g$ and centring values here come from the initial fit.

Thus an individual EIF value generally differs from $C_i$; the extra centring term disappears only after averaging over everyone. Sum each term separately:

$
\begin{aligned}
\frac1n\sum_{i=1}^n\widehat D_i^{\mathrm{init}}
&=\frac1n\sum_{i=1}^n C_i
 +\frac1n\sum_{i=1}^n(\widehat Q_{1,i}-\widehat Q_{0,i})
 -\frac1n\sum_{i=1}^n\widehat\psi_{\mathrm{gcomp}}\\
&=\frac1n\sum_{i=1}^n C_i
 +\widehat\psi_{\mathrm{gcomp}}
 -\frac n n\widehat\psi_{\mathrm{gcomp}}\\
&=\frac1n\sum_{i=1}^n C_i.
\end{aligned}
$

> **Notation recap:** $n=5735$ and the sums cover every patient. In the second line, the average predicted risk difference equals $\widehat\psi_{\mathrm{gcomp}}$ by definition. The last term adds the same centre $n$ times and divides by $n$. These two terms cancel; the average weighted residual may still be non-zero.

Distinguish three objects: $\widehat\psi_{\mathrm{gcomp}}$ is one risk difference estimate from initial $\widehat Q$; $\widehat D^{\mathrm{init}}$ is a function built from initial $\widehat Q$, $\widehat g$ and that centre; averaging its outputs over 5735 patients and adding this to the initial estimate gives **one-step estimation**. For the present standardised risk difference, this one-step estimate is exactly AIPW:

$
\begin{aligned}
\widehat\psi_{\mathrm{one\text{-}step}}=\widehat\psi_{\mathrm{AIPW}}
&=\widehat\psi_{\mathrm{gcomp}}
 +\frac1n\sum_{i=1}^n\widehat D_i^{\mathrm{init}}\\
&=\widehat\psi_{\mathrm{gcomp}}+\frac1n\sum_{i=1}^n C_i\\
&=\frac1n\sum_{i=1}^n
 \{\widehat Q_{1,i}-\widehat Q_{0,i}+C_i\}.
\end{aligned}
$

> **Notation recap:** $\widehat\psi_{\mathrm{one\text{-}step}}$ and $\widehat\psi_{\mathrm{AIPW}}$ are the same final point estimate here; $\widehat\psi_{\mathrm{gcomp}}$ is the initial, uncorrected estimate, $\widehat D_i^{\mathrm{init}}$ the initial EIF value for patient $i$, and $C_i$ that patient’s weighted residual. With $n=5735$, summing and dividing by $n$ averages over all patients. Hats denote fitted or sample-calculated quantities; multiplying risk differences by 100 gives percentage points.

<div class="tmle-eif-aipw-placeholder"></div>

**“One step” means one EIF correction after the initial fits.**

**TMLE starts from the same initial fits and applies the correction to the prediction function.** AIPW’s one-step estimate retains initial predictions and directly yields a corrected number; TMLE retains $\widehat g$, updates $\widehat Q$ along the same $H$, then standardises the updated predictions. Using $H$ as a covariate in a logistic update makes its score $H$ times the updated residual. The next step fits the update size with this regression, bringing the overall average weighted residual close to zero.

<!-- tmle-step:2 -->
### ③ Fit a logistic regression with just one coefficient

“Updating $Q$” means turning **the initial conditional-mean estimate $\widehat Q$ into a new estimated function $\widehat Q^*$**. The update involves three things:

- **Initial model parameters stay fixed.** Every fitted coefficient $\widehat\beta$ of the initial logistic regression keeps its value, as does initial $g$.
- **Add one coefficient to estimate.** The fluctuation parameter $\varepsilon$ controls the shared update size in direction $H$.
- **The new function produces new predictions.** After fitting $\widehat\varepsilon$, initial $Q$ together with this adjustment defines $\widehat Q^*$; updated predictions in the table are evaluations of that new function at each patient’s inputs.

Specifically, use the initial prediction’s logit as a fixed offset, observed mortality $Y$ as response, and $H$ as the sole covariate. Fit a logistic regression **without an additional intercept**. For any input $(a,w)$, the prediction family is:

$
\operatorname{logit}Q_\varepsilon(a,w)
=\underbrace{\operatorname{logit}\widehat Q(a,w)}_{\text{retain the initial model}}
+\underbrace{\varepsilon H(a,w)}_{\text{new adjustment term}}.
$

> **Notation recap:** $a$ is a treatment input, 0 or 1, and $w$ a baseline input; $H(a,w)$ writes Step ②’s $H$ for any such input. The function $Q_\varepsilon$ estimates the conditional mean at a given $\varepsilon$ and returns to $\widehat Q$ when $\varepsilon=0$. Fitting uses observed $(A_i,W_i)$ to obtain individual offsets and $H_{A,i}$, with offset coefficient fixed at 1. The only estimated coefficient, $\varepsilon$, is shared by all patients; $\widehat Q^*=Q_{\widehat\varepsilon}$. Logit acts on predicted probabilities; the regression response is still binary $Y_i$.

The familiar “covariate times residual” score is $H$ times the residual here. Fitting $\varepsilon$ brings its full-sample average close to zero:

$
\frac1n\sum_i H_{A,i}\{Y_i-Q_{\widehat\varepsilon}(A_i,W_i)\}\approx0.
$

> **Notation recap:** $n=5735$; $\widehat\varepsilon$ is the fitted shared coefficient, and braces contain each updated residual. This is the likelihood score equation for $\varepsilon$, with $H$ constructed from the propensity score. The numerical solver stops when the score is within a specified tolerance of zero.

**Why is one shared $\varepsilon$ enough?** We are estimating one population risk difference. Each time we standardise current $Q$, the EIF component “individual predicted risk difference minus overall average” automatically has empirical average zero. The remaining equation to solve by updating is the single mean-zero weighted-residual equation above.

Along the logistic update path used in this example, $H$ is fixed, and varying one $\varepsilon$ continuously changes that summary. The figure below shows the finite root found in this fit. One $\varepsilon$ therefore suffices for this targeted correction. This conclusion depends on choosing a direction matching the EIF, beyond merely having a scalar target. The prediction quality of $Q$ across all baseline profiles still relies on learning and checking the initial regression.

**A shared $\varepsilon$ still gives different individual adjustments.** The amount added to each person’s logit is $\varepsilon$ times their own $H$; its sign and size vary with treatment and baseline characteristics. Compare the first two patients from the same CSV below, then see how the equation approaches zero across all 5735.

<div class="tmle-direction-placeholder"></div>

<!-- tmle-step:3 -->
### ④ Apply the same $\varepsilon$ to both treatment predictions

Step ③ produced the new estimated function $\widehat Q^*$. Now supply each person’s $(1,W_i)$ and $(0,W_i)$ separately to calculate predictions under both treatments. Each evaluation first converts the initial prediction to logit, adds the same $\widehat\varepsilon$ times the relevant $H$, then applies expit to return to probability. Expit is the inverse of logit.

$
\widehat Q^*_{a,i}
=\operatorname{expit}\{\operatorname{logit}\widehat Q_{a,i}
+\widehat\varepsilon H_{a,i}\},\qquad a\in\{0,1\}.
$

> **Notation recap:** $a$ is the treatment level supplied for this evaluation; $\widehat Q_{a,i}$ and $H_{a,i}$ select that arm’s columns. The star marks updated $Q$; both arms share Step ③’s fitted $\widehat\varepsilon$. The update adds on the logit scale and returns a mortality probability between 0 and 1. Observed $W$, $A$, $Y$ and initial $g$ stay fixed.

**Pass to the next step:** two updated mortality-risk columns. Below, compare the operations before and after updating the same patient.

<!-- tmle-step:4 -->
### ⑤ Standardise again using updated $Q$

Return to the original research question. For each person, subtract updated no-RHC risk from updated RHC risk, then average to obtain the TMLE estimate. This point estimate is the average difference between the two predicted risks.

$
\widehat\psi_{\mathrm{TMLE}}=\frac1n\sum_{i=1}^n
(\widehat Q^*_{1,i}-\widehat Q^*_{0,i}).
$

> **Notation recap:** $\widehat\psi_{\mathrm{TMLE}}$ is the final risk difference; $\widehat Q^*_{1,i}$ and $\widehat Q^*_{0,i}$ are Step ④’s updated predictions under both treatments. The average covers all $n=5735$ patients; multiplying by 100 gives percentage points.

**What does making the equation zero accomplish?** Initial $Q$ predicts under both treatments. Observed mortality lets us calculate each record’s “observed outcome minus predicted probability” residual, then examine their aggregate. The EIF’s $H$ weights the residuals for the target population and combines them as “RHC minus no RHC”. This average provides correction information specific to the risk difference.

AIPW adds it to the initial risk difference. TMLE updates $Q$ along $H$ until updated predictions and observed outcomes agree on this weighted check, then recalculates the risk difference. **Zero means this within-sample check is satisfied; whether the estimate is closer to the true target is a separate question requiring the truth for direct assessment.**

<div class="tmle-calibration-placeholder"></div>

**Why is this correction doubly robust?** Follow where the residuals come from:

- **When $Q$ is correct:** among people with the same baseline characteristics and treatment, the prediction equals their mean mortality risk. Such residuals average to zero in the population; multiplying by $H$, determined by treatment and baseline, still gives average zero. Initial standardisation already gives the target, and the population correction does not move it away.
- **When $g$ is correct:** even with biased $Q$, correct inverse probability weights convert observed residuals in each arm into average prediction bias in the common target population. Subtracting the two arms’ biases exactly supplies the missing part of the initial risk difference.
- **When both have errors:** the predictions and the weights used to correct them are both inaccurate, so bias can remain. The remainder involves both $Q$ error and $g$ error: initial $Q$ for AIPW and updated $Q$ for TMLE. This explains why estimating either accurately helps.

These relationships average over a population distribution. AIPW uses the correction directly; TMLE makes updated $Q$ satisfy an equation with the same structure. For estimation, double robustness means that, under the relevant regularity conditions, the estimator approaches the target as sample size grows if at least one of $Q$ and $g$ is consistently estimated. Finite samples also contain sampling error, so one update may move closer to or further from the true value. We can directly verify equations and calculations in these data; double robustness explains why this correction is useful under those conditions.

<details>
<summary>Can bias remain after the equation is zero in a known reference distribution?</summary>

Fix the joint frequencies of sex, RHC and 30-day mortality among these 5735 patients as a **teaching reference distribution**, denoted $P_{\mathrm{ref}}$. Adjusting only for sex, its reference risk difference is directly calculable. Below, all averages use this fixed distribution to isolate the effects of errors in $Q$ and $g$.

“Correct $Q$” uses the actual mortality proportion in each sex–treatment group of the reference distribution; “correct $g$” uses each sex group’s actual RHC proportion. We deliberately change these predictions to create three scenarios. All use the same TMLE update rule.

<div class="tmle-reference-placeholder"></div>

In the third scenario the equation is zero, yet TMLE still differs from the reference target. This shows that **solving the equation and removing all estimation error are different achievements**. The first two scenarios illustrate the population double-robustness relationship: one correct component returns us to the reference target.

The figure’s reference risk difference adjusts only for sex; the actual analysis above adjusts for a fuller $W$, so they target different statistical quantities. $P_{\mathrm{ref}}$ treats this sample’s frequencies as known probabilities; the true population $P_0$ remains unknown. This comparison illustrates the correction mechanism. Accuracy of the actual RHC analysis still depends on model fitting, sampling and causal identification conditions.

<details>
<summary>Review how the reference experiment changes $Q$ and $g$</summary>

Sex is ordered Female, Male. Correct $Q$ uses the four reference mortality proportions. For incorrect $Q$, increase no-RHC predictions by 0.12 and 0.06 respectively, and decrease RHC predictions by 0.06 and 0.12 respectively. Correct $g$ uses the two reference RHC proportions; the first scenario sets both to 0.5, and the third moves each original value halfway towards 0.5. All predictions remain between 0 and 1. Within each update, the initial fitted $Q$ and $g$ stay fixed; only a shared $\varepsilon$ is fitted, then the equation is checked with updated $Q$ and fixed $g$.

</details>
</details>

<!-- tmle-step:5 -->
### ⑥ Use the updated EIF to study uncertainty

After obtaining the point estimate, the EIF has a second use. Substitute updated $Q$, fixed $g$ and the final risk difference into the EIF and calculate one output per patient. It adds a “weighted residual” to “the deviation of the individual predicted risk difference from the overall average”:

$
\widehat D_i=H_{A,i}(Y_i-\widehat Q^*_{A,i})
+(\widehat Q^*_{1,i}-\widehat Q^*_{0,i})-\widehat\psi_{\mathrm{TMLE}}.
$

> **Notation recap:** $\widehat D_i$ is the fitted EIF value for observed record $i$; $\widehat Q^*_{A,i}$ selects the updated prediction under observed $A$, and $H$ still uses initial $g$. The subtracted term is Step ⑤’s overall risk difference. Individual values may be non-zero; their average is close to zero because Step ③ brings the average of the first term close to zero and Step ⑤ centres the last two.

First calculate the **empirical variance** of these values: using the current sample, measure their dispersion around their sample average. In the RHC cohort, subtract the average from each of the 5735 EIF values, square, then average. The denominator here is $n$, or 5735. The sample variance commonly used to estimate population variance without bias uses $n-1$; this tutorial uses variance under the empirical distribution, with denominator $n$. This dispersion provides a scale for estimation error. Under inference conditions including independent observations, sufficient overlap and negligible fitting-error remainders, estimate standard error (SE) as:

$
\overline D=\frac1n\sum_{i=1}^n\widehat D_i,\qquad
\widehat{\mathrm{Var}}(\widehat D)=\frac1n\sum_{i=1}^n(\widehat D_i-\overline D)^2,\qquad
\widehat{\mathrm{SE}}=\sqrt{\frac{\widehat{\mathrm{Var}}(\widehat D)}n}.
$

> **Notation recap:** $\widehat D_i$ is patient $i$’s EIF value and $\overline D$ their sample average; $\widehat{\mathrm{Var}}(\widehat D)$ is the variance calculated above, and $n$ the number of independent observations. Taking the square root gives the risk difference’s standard error. The figure uses this same formula; Chapters 7–8 explain conditions and error control for valid inference.

**The EIF’s two uses now connect:** Steps ②–③ use it to select an update direction and solve an equation; this step uses it to quantify estimation uncertainty. Double robustness guarantees consistency under the relevant conditions; valid interval estimation also needs the inference conditions here.


We use logistic regression for the initial $Q$ and $g$ fits; other suitable prediction methods can also be used at these stages. TMLE’s defining structure is the subsequent targeted update. Chapter 8 maps each step to Julia/R code so that we can inspect the solution process.

Next, temporarily pool the two treatment groups and ask only about mean 30-day mortality among these 5735 patients. The mean’s IF is $y-\mu$: it takes an outcome $y$ and returns its difference from the reference mean $\mu$. **Why does this difference describe how a distributional change affects the mean?** Chapter 1 derives it from a weighted average; Chapter 8 returns to the same cohort to complete AIPW and TMLE step by step.

[Continue to Chapter 1: Deriving the IF from mean mortality →](#chapter-1)

<!-- Temporarily hide the supplementary reading link; retain the material for restoration.
[Supplementary reading: estimation targets, IPW/PSM, GLM scores, IF means and correction equations, and Julia/R workbooks](chapter-0-details.html#chapter-0)
-->


---

# Chapter 1: How do we derive the mean IF from 30-day mortality?

Chapter 0 ended with a question: why is the influence function of the mean $y-\mu$? This chapter derives that rule from the mean itself, then compares it with the familiar logistic regression score.

We continue with the same 5735 RHC cohort patients: 1918 died within 30 days and 3817 survived. For this mean problem, we pool the treatment groups and use only the 30-day outcome. We first calculate the mortality proportion, then change the probability weights of the two outcomes to see how the mean changes. Later, we will use the same approach to derive the IF of the standardised RHC risk difference.

## 0. Define the 30-day mortality outcome

Keep the established coding: original `dth30` values of Yes become $Y=1$, and No become $Y=0$. The values 1 and 0 record vital status at 30 days. If $T$ denotes time from enrolment to death from any cause, in days, we can write this coding as:

$$
Y=\mathbf1\{T\le30\}.
$$

> **Notation recap:** $\mathbf1$ is an indicator function: 1 when the condition holds and 0 otherwise. Capital $T$ is time and $Y$ is the outcome code. Lower-case $t$ will later denote the proportion of probability weight replaced: capital $T$ is measured in days, whereas lower-case $t$ is a dimensionless step size for probability change.

<div class="course-visual" data-visual="survival-window"></div>

The example dates in the figure help us interpret 0 and 1; calculations still use the outcome column in the original CSV. The original study confirmed every patient’s date of death or survival to 180 days, so all 5735 of these 30-day outcomes are known.

<details><summary>Can we determine the 30-day outcome after only 20 days of follow-up?</summary>

If a patient is lost to follow-up after day 20 and no other vital-status source is available, their 30-day outcome should be unknown. Assigning $Y$ requires knowing vital status at day 30. The original study confirmed vital status in these data, so we can directly average the complete outcome column here.

</details>

## 1. Write the mortality proportion as a probability-weighted average

The value 1918/5735 calculated in Chapter 0 is simply the average of this column of zeros and ones. Grouping patients by outcome lets us multiply each group’s size by its outcome code. Distributing the denominator over the two terms shows how each probability enters the mean. A hat marks a result calculated from the current sample:

$$
\begin{aligned}
\widehat\mu&=\frac{3817\times0+1918\times1}{5735}\\
&=0\times\frac{3817}{5735}+1\times\frac{1918}{5735}\\
&=\frac{1918}{5735}\approx0.334438.
\end{aligned}
$$

> **Notation recap:** $\widehat\mu$ is the sample mortality proportion; 3817 and 1918 are counts, and 5735 is the total. Dividing each count by the denominator gives an empirical probability. Displayed decimals are rounded; the fractions retain exact values.

The figure shows the same calculation: one outcome per patient row forms a vector. Adding all 5735 entries and dividing by 5735 returns a scalar, the mortality proportion. The display previews only some patient rows; the average uses the complete outcome column.

<div class="course-visual" data-visual="sample"></div>

We will next change the probability weights, so first fix our starting point. Retain Chapter 0’s empirical distribution $P_n$: each RHC record has weight $1/5735$. In this chapter, use it as the reference distribution $P=P_n$. The survival group therefore has total weight 3817/5735, and the death group 1918/5735.

Below, $\mu$ denotes the mean of this fixed reference; $\widehat\mu$ emphasises an estimate calculated from the sample. Here their numerical values agree, but their roles differ. The true population distribution $P_0$ and its mean remain unknown; for now, we calculate the derivative at a known reference.

Write the death probability as $r$ and the survival probability as $c$. Every subsequent step uses these two exact fractions:

$$
r=\frac{1918}{5735},\quad c=\frac{3817}{5735},\quad r+c=1,\quad\mu=\mathbb E_P[Y]=0c+1r=r.
$$

> **Notation recap:** $r$ is the reference death probability and $c$ the survival probability, found by dividing each count by the total; $\mathbb E_P$ averages with $P$’s weights, and $\mu$ is the result. Every later occurrence of $r$ or $c$ uses these same fractions.

<div class="course-visual" data-visual="distribution"></div>

We write this relationship in functional notation: $\Psi(P)=\mathbb E_P[Y]=\mu$. The input consists of the probabilities of the two outcomes; the output is their weighted average. To find the mean’s IF, ask: when the input probabilities change, how much does the output change?

## 2. Choose a specific probability change

To calculate a rate of change, we need a rule that can be varied continuously. First select death, outcome 1: remove 10% of the weight from each category of the original distribution, then put all the removed weight into category 1.

The survival category starts with $c$ and retains $0.9c$; the death category starts with $r$ and retains $0.9r$. Together, we remove $0.1c+0.1r=0.1(c+r)=0.1$ and add it all to the death category. The new weights are therefore $0.9c$ and $0.9r+0.1$. The figure retains the original weights, removed portions and redistributed result so that you can check each step.

<div class="course-visual" data-visual="replacement"></div>

The patient records stay fixed; we redistribute the probabilities of outcomes 0 and 1. Write $z$ for the outcome receiving the weight, here $z=1$, and $t$ for the proportion removed and replaced, here $t=0.1$. Varying $t$ continuously gives a whole path of distributions.

This 10% is a replacement proportion. The $0.1r$ removed from the death category returns to that category, so the net increase in death probability is only the $0.1c$ transferred from survival, about 6.66 percentage points. If we denote the probability transferred from survival to death by $\delta$, the two quantities satisfy $\delta=tc$. Distinguishing the replacement proportion from the net transfer tells us which quantity we differentiate with respect to.

Still fixing $z=1$, replace 10% by a general $t$. The new probabilities and their sum are:

$$
\begin{aligned}
p_0(t)&=(1-t)c,\\
p_1(t)&=(1-t)r+t,\\
p_0(t)+p_1(t)&=(1-t)c+(1-t)r+t\\
&=(1-t)(c+r)+t\\
&=(1-t)\times1+t=1-t+t=1.
\end{aligned}
$$

> **Notation recap:** $p_0(t),p_1(t)$ are the new probabilities; $c,r$ remain the fixed original probabilities, and $t$ is dimensionless. The last four lines factor out the common term and cancel $-t+t$, verifying that total probability remains 1.

<div class="course-visual" data-visual="mixture"></div>

We can condense this calculation into a distribution formula. A distribution placing all probability at $z$ is a point mass, denoted $\delta_z$; it receives all the weight removed from the original distribution. Thus:

$$
P_t=(1-t)P+t\delta_z.
$$

> **Notation recap:** $P_t$ is the perturbed distribution and $P$ the reference distribution; $\delta_z$ concentrates all its weight at the selected outcome. The subscripted $\delta_z$ denotes a distribution; the unsubscripted $\delta$ above denotes an amount of probability transferred. Here $0\le t\le1$ describes an ordinary mixture: retain $1-t$ of the original distribution and place $t$ at the selected outcome.

## 3. Exactly how much does the mean change along this path?

With the new probabilities, recalculate using the same weighted-average rule. Still taking $z=1$, multiply 0 and 1 by their respective new probabilities, then expand the brackets line by line:

$$
\begin{aligned}
\Psi(P_t)&=0\times[c(1-t)]+1\times[r(1-t)+t]\\
&=0c-0ct+1r-1rt+1t\\
&=0-0+r-rt+t\\
&=r-rt+t\\
&=r+(-r+1)t\\
&=r+(1-r)t=r+ct.
\end{aligned}
$$

> **Notation recap:** $\Psi(P_t)$ is the new risk; $r$ is the original death probability and $c$ is $1-r$. The second line uses the distributive law; the third evaluates multiplication by 0 and 1. Finally, factor $t$ from $-rt+t$. These terms cancel only partly, leaving $ct$.

<div class="course-visual" data-visual="mean-change"></div>

The new mean is now $r+ct$, and the original mean is $r$. Subtract to get the change, then divide by the replacement proportion $t$ to obtain the change per unit replacement proportion. This ratio is the difference quotient:

$$
\begin{aligned}
\Psi(P_t)-\Psi(P)&=(r+ct)-r\\
&=(r-r)+ct=ct,\\
\frac{\Psi(P_t)-\Psi(P)}t&=\frac{ct}t=c\quad(t\ne0),\\
\left.\frac{d}{dt}\Psi(P_t)\right|_{t=0}&=c=1-r.
\end{aligned}
$$

> **Notation recap:** The first bracketed difference is the change; the difference quotient is “change ÷ step size”. The derivative is the limit of this quotient at the starting point. Here the quotient already equals $c$ for every valid non-zero $t$, so taking the limit does not change its value. The actual change is $ct$; the rate per unit $t$ is $c$.

<div class="course-visual" data-visual="rate"></div>

### Does the same rule hold if we choose outcome 0?

So far, we have placed weight in the death category. We could instead place it in the survival category, choosing $z=0$. We want a single formula covering both choices.

Put outcomes 0 and 1 in separate rows, indexed by $j$. Let $y_j$ be the outcome code in that row and $p_j$ its original probability. Retain $(1-t)p_j$ in each row, then add $t$ to the row whose outcome equals $z$. The indicator $\mathbf1\{y_j=z\}$ is 1 in the selected row and 0 in the other, so both new probabilities can be written $(1-t)p_j+t\mathbf1\{y_j=z\}$. Summing over the two rows gives:

$$
\begin{aligned}
\Psi(P_t)&=\sum_j y_j[(1-t)p_j+t\mathbf1\{y_j=z\}]\\
&=\sum_j[y_jp_j-ty_jp_j+ty_j\mathbf1\{y_j=z\}]\\
&=\sum_j y_jp_j-t\sum_j y_jp_j+t\sum_j y_j\mathbf1\{y_j=z\}\\
&=\mu-t\mu+tz\\
&=\mu+t(z-\mu).
\end{aligned}
$$

> **Notation recap:** $\sum_j$ adds the two rows. The first two sums equal the mean $\mu$; the last indicator selects only the chosen row, giving $z$. Since $t$ is the same in both rows, it can be taken outside the sum. Throughout, $\mu$ is the mean of the fixed reference distribution before perturbation.

<div class="course-visual" data-visual="general"></div>

When $z=0$, the death category retains $(1-t)r$, so the net probability transferred from survival to death is $\delta=-tr$; when $z=1$, it is $\delta=tc$. This also explains why the same replacement proportion produces changes of different sizes in the two directions.

The result separates the original mean from its change: $\mu$ is the starting value and $t(z-\mu)$ is the change along the selected path. As before, subtract the original mean and divide by a non-zero step size:

$$
\begin{aligned}
\Psi(P_t)-\Psi(P)
&=[\mu+t(z-\mu)]-\mu\\
&=(\mu-\mu)+t(z-\mu)=t(z-\mu),\\
\frac{\Psi(P_t)-\Psi(P)}{t}
&=\frac{t(z-\mu)}{t}=z-\mu\quad(t\ne0),\\
\lim_{t\to0}\frac{\Psi(P_t)-\Psi(P)}{t}
&=z-\mu.
\end{aligned}
$$

> **Notation recap:** $P$ and its mean $\mu$ stay fixed; $z$ selects the outcome towards which weight moves, and $t$ controls the replacement proportion. The first two lines cancel the original mean; the third cancels $t$ between numerator and denominator when $t$ is non-zero. The quotient no longer depends on $t$, so its limit in the fourth line is the same number. At $t=0$ the distributions coincide and the quotient $0/0$ is undefined; the derivative is the limit of the quotients as the step approaches zero.

## 4. How does the rate of change become an influence function?

We now have two answers to check: replacement towards 0 gives rate $0-\mu=-r$, and replacement towards 1 gives rate $1-\mu=c$. Only the selected outcome $z$ changed; the original distribution $P$ and mean $\mu$ stayed the same.

The influence function (IF) collects these rates into one rule: take an outcome and return the mean’s rate of change when probability weight moves towards it. The input $z$ is 0 or 1, and the output $z-\mu$ is a real number:

$$
D(z;P)=\left.\frac{d}{dt}\Psi((1-t)P+t\delta_z)\right|_{t=0}=z-\mu.
$$

> **Notation recap:** $D(z;P)$ takes a specific outcome $z$ and returns the corresponding rate of change. The $P$ after the semicolon identifies the distribution supplying the reference mean $\mu$. We can also write this rule as $D^{\mathrm{mean}}_P(z)$: either a subscript or a semicolon marks the fixed reference. We differentiate with respect to replacement proportion $t$, holding $z$ fixed.

The derivatives determine both possible function values: 0 maps to $-\mu$, and 1 to $1-\mu$. Together these give “input minus reference mean”. The figure links the derivative results to the function’s two inputs.

<div class="course-visual" data-visual="rate-function"></div>

In the derivation, $z$ emphasises the outcome selected as the replacement direction. When applying the function to a known record, we usually write its observed outcome as $y$. Thus $D(y;P)=y-\mu$ is precisely the rule used in Chapter 0; changing the input letter does not change the calculation. Input 0 gives about −0.3344, and input 1 about +0.6656. The figure places both inputs beside their outputs.

<div class="course-visual" data-visual="influence"></div>

Once the rule is fixed, we can discuss the probability of each output. When drawing a record from the fixed reference distribution $P$, capital $Y$ denotes its as-yet unknown outcome. Substituting it into the same function makes $D(Y;P)=Y-\mu$ a random variable. It equals $c$ with death probability 1918/5735, and $-r$ with survival probability 3817/5735. The figure maps outcome probabilities directly to output probabilities.

<div class="course-visual" data-visual="random-influence"></div>

### Why do two non-zero outputs average to zero?

Now average all possible outputs according to their probabilities. Survival gives the negative value $-r$, and death the positive value $c$; their magnitudes and probabilities differ. Multiply each output by its reference probability, then add:

$$
\begin{aligned}
\mathbb E_P[D(Y;P)]&=c(0-r)+r(1-r)\\
&=0c-cr+r-r^2\\
&=-cr+r-r^2\\
&=r-r(c+r)\\
&=r-r\times1=0.
\end{aligned}
$$

> **Notation recap:** $c,r$ are the two row weights, and the brackets contain the two influence values. After weighting by their respective probabilities, positive and negative contributions cancel exactly because $c+r=1$. Each outcome keeps its own influence value; zero is their weighted average.

<div class="course-visual" data-visual="centering"></div>

The positive and negative parts cancel because each outcome has had the same mean subtracted, calculated with these same probabilities. Writing the two categories as a sum makes the reason clearer:

$$
\begin{aligned}
\mathbb E_P[Y-\mu]&=\sum_jp_j(y_j-\mu)\\
&=\sum_jp_jy_j-\sum_jp_j\mu\\
&=\mu-\mu\sum_jp_j\\
&=\mu-\mu\times1=0.
\end{aligned}
$$

> **Notation recap:** $\mu$ is fixed across rows, so it comes outside the sum. The first term is $\mu$ by the definition of the mean, and the probabilities sum to 1. Centring places the function in the mean-zero space; the next chapter explains its connection to the constraint that probabilities sum to 1.

<div class="course-visual" data-visual="centering-proof"></div>

We must distinguish two ways of averaging the same function. We constructed the rule using $P$, then averaged using $P$’s probabilities, so the centre matched exactly. If we keep the old rule $D(Y;P)=Y-\mu$ but average with $P_t$’s probabilities, we get $\mathbb E_{P_t}D=\Psi(P_t)-\mu=t(z-\mu)$. What remains is the difference between the new and old means.

To obtain mean zero under the new distribution, also change the rule’s centre to the new mean, using $Y-\Psi(P_t)$. The figure first fixes the old function, then updates the averaging weights, and finally updates the function’s centre, showing which object changes each time.

<div class="course-visual" data-visual="reference-distribution"></div>

### How does the same residual connect to the logistic regression score?

Probability replacement has given us the mean IF. Now consider an intercept-only logistic model and derive the same residual by differentiating the log-likelihood. For a binary outcome $Y$, one probability $\mu$ determines the entire Bernoulli distribution. Using an intercept $\beta=\operatorname{logit}(\mu)$ is a different parametrisation of the same family of distributions. Here $0<\mu<1$, so both outcomes have positive probability.

Fix one record’s outcome $y$, write its probability and take its logarithm. The derivative of this log-likelihood with respect to the intercept is the single-observation score:

$$
\begin{aligned}
p_\beta(y)&=\mu^y(1-\mu)^{1-y},\qquad \mu=\frac{e^\beta}{1+e^\beta},\\
\ell(\beta;y)&=y\log\mu+(1-y)\log(1-\mu)\\
&=y\{\beta-\log(1+e^\beta)\}+(1-y)\{-\log(1+e^\beta)\}\\
&=y\beta-\{y+(1-y)\}\log(1+e^\beta)\\
&=y\beta-\log(1+e^\beta),\\
s_\beta(y)&=\frac{\partial\ell(\beta;y)}{\partial\beta}
=y-\frac{e^\beta}{1+e^\beta}=y-\mu.
\end{aligned}
$$

> **Notation recap:** $y$ is fixed at 0 or 1; $\beta$ is the intercept being varied, and $\mu$ changes with $\beta$ through expit. $p_\beta(y)$ gives the probability of this outcome; $\ell$ is its natural logarithm, and $s_\beta$ is one record’s derivative contribution. The third line substitutes the logit parametrisation into both terms; the fifth combines $y+(1-y)=1$. This derivative is with respect to $\beta$, whereas the earlier IF derivation differentiated with respect to replacement proportion $t$.

Thus, at the current reference, both rules return about +0.6656 for a patient who died and about −0.3344 for a survivor. For the score, the sign tells us how increasing the intercept changes that record’s log-likelihood. For the mean IF, it tells us how adding weight towards this outcome changes the death probability.

How far does this equality extend? First sum the scores. Below, $\widehat\beta$ is the intercept-only maximum likelihood estimate and $\widehat\mu=\operatorname{expit}(\widehat\beta)$:

$$
\begin{aligned}
0&=\sum_{i=1}^n s_{\widehat\beta}(y_i)
=\sum_{i=1}^n(y_i-\widehat\mu)\\
&=\sum_{i=1}^n y_i-n\widehat\mu,\\
n\widehat\mu&=\sum_{i=1}^n y_i,\qquad
\widehat\mu=\frac1n\sum_{i=1}^n y_i=\frac{1918}{5735}.
\end{aligned}
$$

> **Notation recap:** $n$ is 5735 patients, and $\sum_i y_i$ is 1918 deaths. Hats mark results fitted from these data. We solve for a total score of zero; each record still contributes its own positive or negative value. The fitted probability equals the sample mean, so the corresponding mean IF also has empirical average zero. The estimation error relative to the unknown true mean $\mu_0$ remains $\widehat\mu-\mu_0$.

### Change the target to distinguish the roles of scores and IFs

Keep the same Bernoulli distribution and logistic score, but suppose we now want to report the logit of death probability, $\beta$. We already calculated the death probability after replacement, $\mu_t=\mu+t(z-\mu)$. Substitute this into the logit and differentiate using the chain rule:

$$
\begin{aligned}
\beta_t&=\log\mu_t-\log(1-\mu_t),\\
D^{\mathrm{logit}}_P(z)
&=\left.\frac{d\beta_t}{dt}\right|_{t=0}\\
&=\left\{\frac1\mu+\frac1{1-\mu}\right\}(z-\mu)\\
&=\frac{(1-\mu)+\mu}{\mu(1-\mu)}(z-\mu)
=\frac{z-\mu}{\mu(1-\mu)}.
\end{aligned}
$$

> **Notation recap:** $\mu_t$ is the death probability after replacement, and $\beta_t$ its logit. $D^{\mathrm{logit}}_P$ denotes the IF for the new target, logit death probability; differentiation is still with respect to $t$. The two minus signs cancel when differentiating $-\log(1-\mu_t)$. The factor $z-\mu$ comes from the already established $d\mu_t/dt$; the preceding bracket converts probability change to logit change.

We can now see that one score can correspond to IFs for different targets. The mean target returns $z-\mu$; the logit target additionally divides by $\mu(1-\mu)$. This divisor is also visible in the likelihood: it is the model’s single-observation Fisher information, denoted $I_\beta$, describing the curvature of the log-likelihood in $\beta$:

$$
\begin{aligned}
\frac{d\mu}{d\beta}
&=\frac{e^\beta(1+e^\beta)-e^{2\beta}}{(1+e^\beta)^2}
=\frac{e^\beta}{(1+e^\beta)^2}=\mu(1-\mu),\\
-\frac{\partial s_\beta(y)}{\partial\beta}&=\mu(1-\mu),\qquad
I_\beta=\mathbb E_P\!\left[-\frac{\partial s_\beta(Y)}{\partial\beta}\right]=\mu(1-\mu),\\
D^{\mathrm{logit}}_P(y)&=I_\beta^{-1}s_\beta(y),\\
D^{\mathrm{mean}}_P(y)&=\frac{d\mu}{d\beta}\,D^{\mathrm{logit}}_P(y)
=\mu(1-\mu)\frac{y-\mu}{\mu(1-\mu)}=y-\mu.
\end{aligned}
$$

> **Notation recap:** $I_\beta$ is information from one observation, and $I_\beta^{-1}$ is its reciprocal in this one-parameter example. The first line uses the quotient rule; the second differentiates the score and changes its sign. The result does not depend on $y$, so taking its expectation leaves it unchanged. The final two lines first convert to the logit target, then multiply by the expit derivative to return to the probability target; the scale factors cancel. These relationships are derived directly in the present Bernoulli model; efficiency in general models is left to Chapters 6–7.

<div class="glm-score-placeholder" data-glm-score="1"></div>

The figure compares calculations for the same patient. Keep the target in view: the enlarged middle value refers to logit death probability, and the last column to death probability. **Predict first: if we report logit death probability, do the absolute IF values increase for both outcomes? Why do their signs stay the same?**

<details><summary>Check the answer and re-express the model using the mean parameter</summary>

Yes. Here $0<\mu(1-\mu)\le1/4$, so dividing by it increases the absolute values, while its positive sign preserves theirs. The two columns measure targets on different scales.

Alternatively, keep death probability as the target but use $\mu$ itself as the likelihood parameter. The chain rule changes the score’s numerical value:

$$
\begin{aligned}
s_\mu(y)&=\frac{\partial}{\partial\mu}\{y\log\mu+(1-y)\log(1-\mu)\}\\
&=\frac y\mu-\frac{1-y}{1-\mu}
=\frac{y(1-\mu)-\mu(1-y)}{\mu(1-\mu)}\\
&=\frac{y-y\mu-\mu+\mu y}{\mu(1-\mu)}
=\frac{y-\mu}{\mu(1-\mu)}.
\end{aligned}
$$

> **Notation recap:** $s_\mu$ differentiates with respect to probability parameter $\mu$; $s_\beta$ differentiates with respect to logit parameter $\beta$. Both describe the same Bernoulli family, but their unit steps differ, so their values differ. The mean IF is still determined by the same probability replacement and mean target, giving $y-\mu$.

</details>

### From a regression parameter score to a distributional path score

Finally, write “change the intercept slightly” as a distributional change. Take another path: the intercept changes from $\beta$ to $\beta+t$, so death probability becomes $\mu_t=\operatorname{expit}(\beta+t)$. This paragraph reuses $t$ for an **increment on the logit scale**; this path differs from the earlier replacement path.

The path score is the derivative of log probability along this path. Since $d(\beta+t)/dt=1$, it equals the intercept score above. The target’s rate of change is the expit derivative:

$$
\begin{aligned}
S_{\mathrm{logit}}(y)
&=\left.\frac{d}{dt}\log p_{\beta+t}(y)\right|_{t=0}
=s_\beta(y)=y-\mu,\\
\left.\frac{d\mu_t}{dt}\right|_{t=0}&=\mu(1-\mu),\\
\mathbb E_P[D^{\mathrm{mean}}_P(Y)S_{\mathrm{logit}}(Y)]
&=(1-\mu)(-\mu)(-\mu)+\mu(1-\mu)(1-\mu)\\
&=\mu(1-\mu)\{\mu+(1-\mu)\}
=\mu(1-\mu).
\end{aligned}
$$

> **Notation recap:** $S$ describes this specified logit path, and $D$ describes the mean target. Expectation uses the fixed $P$ probabilities: weight $1-\mu$ for $Y=0$ and weight $\mu$ for $Y=1$. Multiplying the two functions pointwise and averaging gives the target’s rate of change along this path. This is a concrete calculation of “pairing a gradient with a direction” in a familiar regression model.

<div class="glm-score-placeholder" data-glm-score="pairing"></div>

Three ideas now connect: changing a regression parameter creates a path of distributions; the score describes that path; and multiplying the IF by the score and averaging under the reference distribution gives the target’s path derivative. We have checked the logit path. The next chapter returns to probability replacement, derives its score from its own probability formula and checks the same pairing.

## 5. Predict with the derived rule, then check against the original definition

We now have two calculation routes: average directly with the new probabilities, or add $t(z-\mu)$ to the original mean. First choose $z=1$, $t=0.1$ and predict the increase in mortality and decrease in 30-day survival. Then use the experiment below to check. Afterwards, return to the earlier object diagram, set its replacement proportion $t$ to 0 and compare the change, difference quotient and derivative.

<div class="lab" data-lab="1"></div>

<details><summary>Exercise 1 answer: concentrate 20% of the weight at Y=1</summary>

The new risk is $\mu+0.2(1-\mu)=r+0.2c$; the change is $0.2c$ and the rate is $c$. The figure also calculates the mean directly with the new probabilities, allowing you to cross-check.

<div class="course-visual" data-visual="exercise-one"></div>

</details>

<details><summary>Exercise 2 answer: concentrate 80% of the weight at Y=0</summary>

The new risk is $\mu+0.8(0-\mu)=0.2r$; the change is $-0.8r$ and the rate is $-r$. The original death category retains 20% of its weight.

<div class="course-visual" data-visual="exercise-two"></div>

</details>

## 6. What changes if we report 30-day survival probability?

The experiment also showed survival probability. It sums to 1 with death probability, so any increase in death probability produces an equal decrease in survival probability. Write 30-day survival probability as $S(30)=1-\mu$. Along the same path, new survival minus old survival is $[1-\Psi(P_t)]-[1-\mu]=-t(z-\mu)$, so its IF is $-(z-\mu)$. Here $S(30)$ means survival probability at day 30. The next chapter also uses $S$ for a path score; its arguments and surrounding formula distinguish the two uses.

<div class="course-visual" data-visual="survival-complement"></div>

<details><summary>Why is there no approximation remainder for the mean? Can t be negative?</summary>

Revisit the expansion in §3: outcome values stay fixed, and changes enter only through their probability weights, each linear in $t$. Their weighted sum therefore contains only a constant and a term linear in $t$, making $\Psi(P_t)=\mu+t(z-\mu)$ exact. This is linearity of the mean in probability weights: change equals step size times rate exactly. Later conditional means contain quotients whose denominators also change along the path, requiring us to distinguish first-order terms from remainders.

This chapter’s outcomes are only 0 and 1, with strictly positive reference probabilities $c$ and $r$, so the point-mass replacement path permits differentiation near the starting point. With $z=1$, both probabilities are strictly positive for $-r/c<t<1$; including endpoints permits zero probabilities. Negative $t$ adjusts probabilities in the opposite direction; the ordinary “remove and replace” mixture interpretation applies for $t$ between 0 and 1.

For similar paths in continuous distributions, regular submodel conditions need separate checking: adding a point mass may change the support and density properties. Chapter 5 uses smooth score directions allowed by the model to express and check derivatives.

<div class="course-visual" data-visual="boundary"></div>

</details>

## 7. Recalculate the mean and influence function in code

<div class="lesson-language-control"><label>Code language in this chapter <select data-lesson-language><option value="julia">Julia operations</option><option value="r">R operations</option></select></label><span>Use the same RHC data and the same mathematical objectives</span></div>

<div class="lesson-code-panel" data-lesson-panel="julia">


The code now follows the mathematical steps: read outcomes from the original CSV and calculate the reference mean; select $z$ and $t$ and recalculate the perturbed mean; finally evaluate each IF value and check the weighted average. The input remains the same RHC data, and the outputs match the quantities calculated above.

The code panel offers Julia and R scripts, each with a line-by-line explanation. First follow the inputs, operations and outputs, then inspect unfamiliar syntax; the module handles CSV reading details such as quotation marks. Every line can be linked back to a mean, probability replacement or centring operation above.

The input is [the same original RHC CSV](../shared/data/public/rhc/source.csv). For reading and fitting definitions, see [RHCCase.jl](../shared/julia/RHCCase.jl). The complete [chapter script](julia/rhc_lessons/ch01.jl) is editable; all scripts can be run from other working directories, as long as the relative positions of en/ and shared/ are retained.

```sh
julia --startup-file=no --history-file=no en/julia/rhc_lessons/ch01.jl
```

```julia
# Chapter 1: read original public RHC records and compute mean 30-day mortality.
# include executes another file to make its reading functions available here.
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
# using brings exported module functions into the current script.
using .RHCCase
# DEFAULT_SOURCE is the absolute path to the original CSV frozen with this project.
rows = read_rhc_rows(DEFAULT_SOURCE)
# One row represents one patient; no exclusions, imputation or resampling occur here.
observations = rhc_observations(rows)
# length counts patients, corresponding to n in the formula.
n = length(observations)
# Patient outcome Y: 1 when original dth30 is Yes, otherwise 0.
y = [person.y for person in observations]
# sum adds binary indicators, so events counts deaths.
events = sum(y)
# Division / gives the sample mean: μ̂ = (Y₁ + … + Yₙ) / n.
risk = events / n
# Each patient’s influence value: Dᵢ = Yᵢ − μ̂.
influence = y .- risk
# .- subtracts elementwise; sum then adds the influence values.
mean_influence = sum(influence) / n
# Aggregate the same original records by (X,A,Y) for subsequent derivations.
data = rhc_data(DEFAULT_SOURCE)
# mean_if weights by cell counts and should agree with the patient-level result above.
check = mean_if(data)
# Require grouped and patient-level risks to agree and the patient influence mean to be near zero.
@assert isapprox(check.risk, risk) && abs(mean_influence) < 1e-12
println("Original patient count n = ", n, "; 30-day deaths = ", events)
println("Sample death risk μ̂ = ", risk)
println("IF for Y=0 = ", -risk, "; IF for Y=1 = ", 1-risk)
println("Patient-level IF mean = ", mean_influence, "; IF mean weighted by group counts = ", check.mean)
# This is the full cohort’s descriptive mean; this chapter does not yet estimate treatment effects.
```


<details><summary>Line-by-line explanation: input, operation and output</summary>

### Code line 3

```julia
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
```

<div class="rhc-julia-object-map" data-julia-line="3" data-operation="setup"><div class="ov-expression" role="group" aria-label="The input object at line 3 of code"><strong>Input objects</strong> <span>No existing data objects are read; this line loads definitions.</span></div><div class="ov-expression" role="group" aria-label="The output object of line 3 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="rhc-module" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Module for reading data and calculating results</strong><code>RHCCase</code><small>Type: Module</small><small>Dimensions: Modules, not patient data</small></span></div></div>

<p class="rhc-julia-no-formula">This line prepares names or definitions rather than calculating a statistic.</p>

<p class="rhc-julia-operation" data-julia-operation="3">include reads and executes another Julia file. joinpath joins path components; @__DIR__ identifies the current script directory. Three .. components move up three levels to shared RHCCase.jl. This loads function definitions without calculating patient means.</p>

### Code line 5

```julia
using .RHCCase
```

<div class="rhc-julia-object-map" data-julia-line="5" data-operation="setup"><div class="ov-expression" role="group" aria-label="The input object at line 5 of code"><strong>Input objects</strong> <span class="ov-object ov-slate" data-object="rhc-module" data-direction="input" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Module for reading data and calculating results</strong><code>RHCCase</code><small>Type: Module</small><small>Dimensions: Modules, not patient data</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 5 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="rhc-exports" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Exported functions and constants available to this script</strong><code>using .RHCCase</code><small>Type: export name binding</small><small>Dimensions: A collection of names</small></span> <span class="ov-object ov-slate" data-object="source-path" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Location of the frozen original CSV</strong><code>DEFAULT_SOURCE</code><small>Type: String</small><small>Dimensions: a file path string</small></span></div></div>

<p class="rhc-julia-no-formula">This line prepares names or definitions rather than calculating a statistic.</p>

<p class="rhc-julia-operation" data-julia-operation="5">using brings the loaded RHCCase module’s exported names and DEFAULT_SOURCE into the current script. The leading dot identifies a local module. DEFAULT_SOURCE is a CSV path, not patient data.</p>

### Code line 7

```julia
rows = read_rhc_rows(DEFAULT_SOURCE)
```

<div class="rhc-julia-object-map" data-julia-line="7" data-operation="read"><div class="ov-expression" role="group" aria-label="The input object at line 7 of code"><strong>Input objects</strong> <span class="ov-object ov-slate" data-object="source-path" data-direction="input" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Location of the frozen original CSV</strong><code>DEFAULT_SOURCE</code><small>Type: String</small><small>Dimensions: a file path string</small></span> <span class="ov-object ov-slate" data-object="rhc-exports" data-direction="input" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Exported functions and constants available to this script</strong><code>using .RHCCase</code><small>Type: export name binding</small><small>Dimensions: A collection of names</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 7 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="raw-records" data-direction="output" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Original patient records</strong><code>rows</code><small>Type: Vector{Dict{String,String}}</small><small>Dimensions: n dictionaries, each retaining 63 original fields</small></span></div></div>

$$
\mathrm{rows}=(r_1,\ldots,r_n)
$$

<p class="rhc-julia-operation" data-julia-operation="7">read_rhc_rows reads the original CSV; = stores the returned records as rows. Each entry is one patient’s field dictionary, with original values initially stored as strings. The path and records are separate objects. The reader checks fields and patient identifiers and preserves every original row.</p>

### Code line 9

```julia
observations = rhc_observations(rows)
```

<div class="rhc-julia-object-map" data-julia-line="9" data-operation="encode"><div class="ov-expression" role="group" aria-label="The input object at line 9 of code"><strong>Input objects</strong> <span class="ov-object ov-slate" data-object="raw-records" data-direction="input" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Original patient records</strong><code>rows</code><small>Type: Vector{Dict{String,String}}</small><small>Dimensions: n dictionaries, each retaining 63 original fields</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 9 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="observations" data-direction="output" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Three binary labels for each patient</strong><code>observations</code><small>Type: Vector{NamedTuple{(:x,:a,:y),Tuple{Int,Int,Int}}}</small><small>Dimensions: n (x,a,y) records; in the same order as the original rows</small></span></div></div>

$$
O_i=(X_i,A_i,Y_i),\quad Y_i=\mathbf1\{\mathrm{dth30}_i=\mathrm{Yes}\}
$$

<p class="rhc-julia-operation" data-julia-operation="9">Pass rows to rhc_observations. In original row order, encode sex, swang1 and dth30 as integers x, a and y. Each result is a named (x,a,y) record; these are observed labels, not model predictions, and no patients are removed.</p>

### Code line 11

```julia
n = length(observations)
```

<div class="rhc-julia-object-map" data-julia-line="11" data-operation="reduce"><div class="ov-expression" role="group" aria-label="The input object at line 11 of code"><strong>Input objects</strong> <span class="ov-object ov-slate" data-object="observations" data-direction="input" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Three binary labels for each patient</strong><code>observations</code><small>Type: Vector{NamedTuple{(:x,:a,:y),Tuple{Int,Int,Int}}}</small><small>Dimensions: n (x,a,y) records; in the same order as the original rows</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 11 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="sample-size" data-direction="output" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>total number of patients</strong><code>n</code><small>Type: Int</small><small>Dimensions: One integer scalar</small></span></div></div>

$$
n=\operatorname{length}(O_1,\ldots,O_n)
$$

<p class="rhc-julia-operation" data-julia-operation="11">length counts the patient records in observations and assigns the integer count to n. The container has n records; n itself is one scalar.</p>

### Code line 13

```julia
y = [person.y for person in observations]
```

<div class="rhc-julia-object-map" data-julia-line="13" data-operation="extract"><div class="ov-expression" role="group" aria-label="The input object at line 13 of code"><strong>Input objects</strong> <span class="ov-object ov-slate" data-object="observations" data-direction="input" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Three binary labels for each patient</strong><code>observations</code><small>Type: Vector{NamedTuple{(:x,:a,:y),Tuple{Int,Int,Int}}}</small><small>Dimensions: n (x,a,y) records; in the same order as the original rows</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 13 of code"><strong>Output objects</strong> <span class="ov-object ov-blue" data-object="outcomes" data-direction="output" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Observed 30-day death indicators</strong><code>y</code><small>Type: Vector{Int}</small><small>Dimensions: n-element vector; one 0/1 indicator per patient</small></span></div></div>

$$
\mathbf Y=(Y_1,\ldots,Y_n)^\mathsf T
$$

<p class="rhc-julia-operation" data-julia-operation="13">Square brackets create a vector. for person in observations visits each patient; person.y reads that record’s y field. The resulting y vector preserves patient order and has length n. Y=0/1 are outcome labels, not array indices.</p>

### Code line 15

```julia
events = sum(y)
```

<div class="rhc-julia-object-map" data-julia-line="15" data-operation="reduce"><div class="ov-expression" role="group" aria-label="The input object at line 15 of code"><strong>Input objects</strong> <span class="ov-object ov-blue" data-object="outcomes" data-direction="input" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Observed 30-day death indicators</strong><code>y</code><small>Type: Vector{Int}</small><small>Dimensions: n-element vector; one 0/1 indicator per patient</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 15 of code"><strong>Output objects</strong> <span class="ov-object ov-blue" data-object="events" data-direction="output" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Number of deaths within 30 days</strong><code>events</code><small>Type: Int</small><small>Dimensions: One integer scalar</small></span></div></div>

$$
M=\sum_{i=1}^nY_i
$$

<p class="rhc-julia-operation" data-julia-operation="15">sum adds all n binary entries in y: a death contributes 1 and survival contributes 0. events is one integer scalar containing the total death count.</p>

### Code line 17

```julia
risk = events / n
```

<div class="rhc-julia-object-map" data-julia-line="17" data-operation="divide"><div class="ov-expression" role="group" aria-label="The input object at line 17 of code"><strong>Input objects</strong> <span class="ov-object ov-blue" data-object="events" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Number of deaths within 30 days</strong><code>events</code><small>Type: Int</small><small>Dimensions: One integer scalar</small></span> <span class="ov-object ov-slate" data-object="sample-size" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>total number of patients</strong><code>n</code><small>Type: Int</small><small>Dimensions: One integer scalar</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 17 of code"><strong>Output objects</strong> <span class="ov-object ov-blue" data-object="risk" data-direction="output" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Sample mortality proportion in the full cohort</strong><code>risk</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span></div></div>

$$
\widehat\mu=\frac{M}{n}=\frac1n\sum_{i=1}^nY_i
$$

<p class="rhc-julia-operation" data-julia-operation="17">Divide the death count events by the cohort size n and assign the scalar result to risk. This is the observed mortality proportion for the full cohort.</p>

### Code line 19

```julia
influence = y .- risk
```

<div class="rhc-julia-object-map" data-julia-line="19" data-operation="broadcast"><div class="ov-expression" role="group" aria-label="The input object at line 19 of code"><strong>Input objects</strong> <span class="ov-object ov-blue" data-object="outcomes" data-direction="input" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Observed 30-day death indicators</strong><code>y</code><small>Type: Vector{Int}</small><small>Dimensions: n-element vector; one 0/1 indicator per patient</small></span> <span class="ov-object ov-blue" data-object="risk" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Sample mortality proportion in the full cohort</strong><code>risk</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 19 of code"><strong>Output objects</strong> <span class="ov-object ov-orange" data-object="influence" data-direction="output" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Patient-level influence values for the mean</strong><code>influence</code><small>Type: Vector{Float64}</small><small>Dimensions: n-element vector, negative numbers allowed</small></span></div></div>

$$
D_i=Y_i-\widehat\mu,\quad\mathbf D=\mathbf Y-\widehat\mu\mathbf1_n
$$

<p class="rhc-julia-operation" data-julia-operation="19">The dot in .- means elementwise subtraction (broadcasting): subtract the same scalar risk from every entry of y. influence retains n entries: Y=0 gives -risk and Y=1 gives 1-risk. No averaging occurs here.</p>

### Code line 21

```julia
mean_influence = sum(influence) / n
```

<div class="rhc-julia-object-map" data-julia-line="21" data-operation="reduce"><div class="ov-expression" role="group" aria-label="The input object at line 21 of code"><strong>Input objects</strong> <span class="ov-object ov-orange" data-object="influence" data-direction="input" data-dim="n" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Patient-level influence values for the mean</strong><code>influence</code><small>Type: Vector{Float64}</small><small>Dimensions: n-element vector, negative numbers allowed</small></span> <span class="ov-object ov-slate" data-object="sample-size" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>total number of patients</strong><code>n</code><small>Type: Int</small><small>Dimensions: One integer scalar</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 21 of code"><strong>Output objects</strong> <span class="ov-object ov-orange" data-object="mean-influence" data-direction="output" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Mean patient-level influence value</strong><code>mean_influence</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span></div></div>

$$
\overline D=\frac1n\sum_{i=1}^nD_i=\frac1n\sum_{i=1}^n(Y_i-\widehat\mu)\approx0
$$

<p class="rhc-julia-operation" data-julia-operation="21">sum adds all n influence values and / n divides by the cohort size, producing scalar mean_influence. Positive and negative contributions cancel in the mean. It is exactly 0 in real arithmetic, allowing small floating-point errors; individual influence values need not be 0.</p>

### Code line 23

```julia
data = rhc_data(DEFAULT_SOURCE)
```

<div class="rhc-julia-object-map" data-julia-line="23" data-operation="aggregate"><div class="ov-expression" role="group" aria-label="The input object at line 23 of code"><strong>Input objects</strong> <span class="ov-object ov-slate" data-object="source-path" data-direction="input" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Location of the frozen original CSV</strong><code>DEFAULT_SOURCE</code><small>Type: String</small><small>Dimensions: a file path string</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 23 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="coarse-data" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Counts for the same patients, grouped into eight cells</strong><code>data</code><small>Type: NamedTuple</small><small>Dimensions: Record with named fields; data.cells has eight entries</small></span></div></div>

$$
n_j=\#\{i:(X_i,A_i,Y_i)=(x_j,a_j,y_j)\},\quad\sum_{j=1}^8n_j=n
$$

<p class="rhc-julia-operation" data-julia-operation="23">rhc_data rereads the original CSV and counts patients in each (x,a,y) cell. It returns a record whose data.cells field contains eight cell counts; data.n is the total patient count, 5735.</p>

### Code line 25

```julia
check = mean_if(data)
```

<div class="rhc-julia-object-map" data-julia-line="25" data-operation="validate"><div class="ov-expression" role="group" aria-label="The input object at line 25 of code"><strong>Input objects</strong> <span class="ov-object ov-slate" data-object="coarse-data" data-direction="input" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Counts for the same patients, grouped into eight cells</strong><code>data</code><small>Type: NamedTuple</small><small>Dimensions: Record with named fields; data.cells has eight entries</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 25 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="mean-check" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Grouped mean calculation returned by mean_if</strong><code>check</code><small>Type: NamedTuple{(:risk,:values,:mean)}</small><small>Dimensions: Record containing two scalars and an eight-element vector</small></span> <span class="ov-object ov-blue" data-object="check-risk" data-direction="output" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Mortality proportion weighted by cell counts</strong><code>check.risk</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span> <span class="ov-object ov-orange" data-object="check-values" data-direction="output" data-dim="8" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Mean influence value for each (x,a,y) cell</strong><code>check.values</code><small>Type: Vector{Float64}</small><small>Dimensions: Eight-element vector; one value per cell</small></span> <span class="ov-object ov-orange" data-object="check-mean" data-direction="output" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Mean influence value weighted by cell counts</strong><code>check.mean</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span></div></div>

$$
\mathrm{check.risk}=\sum_{j=1}^8\frac{n_j}{n}y_j,\quad\mathrm{check.mean}=\sum_{j=1}^8\frac{n_j}{n}(y_j-\widehat\mu)
$$

<p class="rhc-julia-operation" data-julia-operation="25">mean_if(data) calculates the mortality proportion and mean influence value from the cell counts, storing the returned record in check. check.risk is a scalar mortality proportion; check.values contains eight cell-specific influence values; check.mean is their count-weighted mean.</p>

### Code line 27

```julia
@assert isapprox(check.risk, risk) && abs(mean_influence) < 1e-12
```

<div class="rhc-julia-object-map" data-julia-line="27" data-operation="assert"><div class="ov-expression" role="group" aria-label="The input object at line 27 of code"><strong>Input objects</strong> <span class="ov-object ov-blue" data-object="check-risk" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Mortality proportion weighted by cell counts</strong><code>check.risk</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span> <span class="ov-object ov-blue" data-object="risk" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Sample mortality proportion in the full cohort</strong><code>risk</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span> <span class="ov-object ov-orange" data-object="mean-influence" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Mean patient-level influence value</strong><code>mean_influence</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 27 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="assertion-result" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Check that must pass before execution continues</strong><code>@assert</code><small>Type: Nothing (throws an AssertionError if the condition fails)</small><small>Dimensions: No new statistical value</small></span></div></div>

$$
\mathrm{check.risk}\approx\widehat\mu\quad\land\quad|\overline D|<10^{-12}
$$

<p class="rhc-julia-operation" data-julia-operation="27">@assert requires both conditions joined by &amp;&amp; to be true. isapprox compares the grouped and patient-level risks using its default tolerance; abs checks that the patient influence mean is below 1e-12. Failure stops execution. This line does not directly check check.mean.</p>

### Code line 28

```julia
println("Original patient count n = ", n, "; 30-day deaths = ", events)
```

<div class="rhc-julia-object-map" data-julia-line="28" data-operation="print"><div class="ov-expression" role="group" aria-label="The input object at line 28 of code"><strong>Input objects</strong> <span class="ov-object ov-slate" data-object="sample-size" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>total number of patients</strong><code>n</code><small>Type: Int</small><small>Dimensions: One integer scalar</small></span> <span class="ov-object ov-blue" data-object="events" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Number of deaths within 30 days</strong><code>events</code><small>Type: Int</small><small>Dimensions: One integer scalar</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 28 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="terminal-output" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Display of previously calculated results</strong><code>println(...)</code><small>Type: Nothing (side effect is writing to the terminal)</small><small>Dimensions: Terminal text</small></span></div></div>

$$
n,\quad M
$$

<p class="rhc-julia-operation" data-julia-operation="28">println displays the quoted labels followed by existing n and events, then starts a new line. Commas separate display arguments without summing them. These are the existing cohort and death counts; no new estimate is computed.</p>

### Code line 29

```julia
println("Sample death risk μ̂ = ", risk)
```

<div class="rhc-julia-object-map" data-julia-line="29" data-operation="print"><div class="ov-expression" role="group" aria-label="The input object at line 29 of code"><strong>Input objects</strong> <span class="ov-object ov-blue" data-object="risk" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Sample mortality proportion in the full cohort</strong><code>risk</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 29 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="terminal-output" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Display of previously calculated results</strong><code>println(...)</code><small>Type: Nothing (side effect is writing to the terminal)</small><small>Dimensions: Terminal text</small></span></div></div>

$$
\widehat\mu
$$

<p class="rhc-julia-operation" data-julia-operation="29">println displays the existing scalar risk after its label and starts a new line. It does not recalculate the mean.</p>

### Code line 30

```julia
println("IF for Y=0 = ", -risk, "; IF for Y=1 = ", 1-risk)
```

<div class="rhc-julia-object-map" data-julia-line="30" data-operation="print"><div class="ov-expression" role="group" aria-label="The input object at line 30 of code"><strong>Input objects</strong> <span class="ov-object ov-blue" data-object="risk" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Sample mortality proportion in the full cohort</strong><code>risk</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 30 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="terminal-output" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Display of previously calculated results</strong><code>println(...)</code><small>Type: Nothing (side effect is writing to the terminal)</small><small>Dimensions: Terminal text</small></span></div></div>

$$
D(0)=-\widehat\mu,\quad D(1)=1-\widehat\mu
$$

<p class="rhc-julia-operation" data-julia-operation="30">println displays the mean-target influence value for each possible outcome: -risk means 0-risk, while 1-risk subtracts the same centre from 1. These are the two function values at Y=0 and Y=1, rather than patient-level predictions or a recalculated n-element vector.</p>

### Code line 31

```julia
println("Patient-level IF mean = ", mean_influence, "; IF mean weighted by group counts = ", check.mean)
```

<div class="rhc-julia-object-map" data-julia-line="31" data-operation="print"><div class="ov-expression" role="group" aria-label="The input object at line 31 of code"><strong>Input objects</strong> <span class="ov-object ov-orange" data-object="mean-influence" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Mean patient-level influence value</strong><code>mean_influence</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span> <span class="ov-object ov-orange" data-object="check-mean" data-direction="input" data-dim="1" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Mean influence value weighted by cell counts</strong><code>check.mean</code><small>Type: Float64</small><small>Dimensions: One numeric scalar</small></span></div><div class="ov-expression" role="group" aria-label="The output object of line 31 of code"><strong>Output objects</strong> <span class="ov-object ov-slate" data-object="terminal-output" data-direction="output" data-dim="container-or-no-value" style="display:inline-flex;flex-direction:column;gap:4px;padding:8px;max-width:100%;overflow-wrap:anywhere;vertical-align:top"><strong>Display of previously calculated results</strong><code>println(...)</code><small>Type: Nothing (side effect is writing to the terminal)</small><small>Dimensions: Terminal text</small></span></div></div>

$$
\overline D,\quad\sum_{j=1}^8(n_j/n)D_j
$$

<p class="rhc-julia-operation" data-julia-operation="31">println displays mean_influence and check.mean under separate labels. The first averages patient-level influence values; the second uses group-count weights. The dot reads the check.mean field; commas separate text and values. No further averaging or equality assertion occurs here.</p>


</details>

<details><summary>Recalculate this chapter’s illustrated operations in Julia</summary>

The [object operations workbook](julia/object_ch01.jl) continues the original-data reading steps above and explains probability vectors, broadcasting, replacement paths, rates of change and centring line by line. The input is the same original RHC CSV; outputs include vector and scalar checks. It preserves original patient row order, using `support_y` for the two outcome rows 0 and 1, and `y` for individual patient outcomes. This distinguishes “weighting two outcome categories” from “averaging 5735 patients”.

```sh
julia --startup-file=no --history-file=no en/julia/object_ch01.jl
```

</details>



</div>

<div class="lesson-code-panel" data-lesson-panel="r" hidden>

R can run one line in the console or a complete file with Rscript. Before copying code directly into the console, set the working directory to the project root. A complete script run through source() or Rscript locates the project automatically. Read the expression on the right before the name on the left: `risk <- events / n` first computes the division, then stores the result as risk. The name points to the object; the same name can receive the result of a new calculation.

| Syntax | What it does | Output shape |
| --- | --- | --- |
| `c(0, 1)` | Combine two numbers | Vector of length 2 |
| `observations$y` | Read a data-frame column by name | Vector of length n |
| `length(y)` / `nrow(rows)` | Count vector elements / data rows | An integer |
| `x[1]` / `q[1, 2]` | Select vector or matrix elements by position, starting at 1 | A number |
| `*`, `/`, `+`, `-` | Calculate element by element; a scalar can operate on a whole column | Usually the vector length is preserved |
| `%*%` | Matrix multiplication | For an m×k matrix times a k×p matrix: m×p |
| `sum(x)` / `mean(x)` | Sum or average a complete vector | A number |
| `==` / `<-` | Test equality / assign a value | Logical object / saved object |
| `function(v)` / `vapply(...)` | Define a function / apply a function to each element | Determined by the function and declared return type |
| `stopifnot(...)` | Requires all check conditions to be met | Continue if passed, stop if failed |

For an R numeric vector of length n, `dim(y)` is usually NULL; n×1 is a teaching representation of a column. A data frame stores different variables in columns; a matrix stores numeric values of one type; a list may hold design matrices, prediction vectors and scalars together. Every calculation uses the full cohort; the two outcome support points and the n patient records are aligned separately.

The input is [the same original public RHC CSV](../shared/data/public/rhc/source.csv). Shared definitions: [RHCCase.R](../shared/R/RHCCase.R) and [EIFCase.R](../shared/R/EIFCase.R). [Complete R script](R/ch01.R) uses base R and runs from any working directory if you retain the project directory structure.

```sh
Rscript --vanilla en/R/ch01.R
```

```r
# Chapter 1: continuous introductory R workbook on means, paths and influence values
# Input: frozen original public RHC CSV; output: chapter objects and terminal checks; base R only.
# commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
# sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
# if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch01.R")
# length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.
stopifnot(length(script_path) == 1L, !is.na(script_path))
# dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
# source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.
source(file.path(project_root, "shared", "R", "RHCCase.R"))
# read_rhc_rows reads the official original CSV. The shared file locates DEFAULT_SOURCE. rows is a 5735×63 data.frame; each row represents the same original patient.
rows <- read_rhc_rows(DEFAULT_SOURCE)
# Encode the original sex, swang1 and dth30 fields as the binary columns x, a and y. Return an n×3 data.frame in the original row order.
observations <- rhc_observations(rows)
# $ selects column y by name. This numeric vector has length n: 1 means death within 30 days, and 0 means survival at 30 days. Its dim is NULL.
y <- observations$y
# length counts vector elements, returning the integer cohort size n=5735. For a data.frame, nrow counts patient rows while length counts variable columns.
n <- length(y)
# sum adds the n binary indicators, returning the death-count scalar 1918. <- assigns the result on the right to the name on the left.
events <- sum(y)
# / divides the death count by the cohort size. Dividing these two scalars gives one sample 30-day mortality risk on the probability scale.
risk <- events / n
# c combines 0 and 1 into a two-element numeric vector. These entries are the two possible outcomes, rather than individual patient records.
support_y <- c(0, 1)
# function(v) defines a rule for one outcome category. == compares each patient’s outcome with v; mean treats TRUE as 1 and FALSE as 0. vapply applies this rule to both categories and requires one number per result, giving p length two.
p <- vapply(support_y, function(v) mean(y == v), numeric(1))
# * multiplies each outcome support value by its probability; sum reduces the two products to one mean. The fixed reference is the empirical outcome distribution.
mu <- sum(p * support_y)
# Choose outcome 1 to receive the added probability mass. z is one numeric scalar and must be either 0 or 1.
z <- 1
# Set the dimensionless replacement proportion to 0.1. With t between 0 and 1, the calculation changes probability weights rather than patients’ recorded y values.
t <- 0.1
# %in% checks membership in the allowed outcome categories. is.finite excludes NA and infinity; stopifnot checks that the path step is valid.
stopifnot(z %in% support_y, is.finite(t), t >= 0, t <= 1)
# == compares both categories with the selected outcome; as.numeric converts FALSE/TRUE to 0/1, giving a two-element point-mass probability vector.
point <- as.numeric(support_y == z)
# Evaluate the parentheses first. A scalar scales the vector element by element; adding the two components gives a new two-element probability vector.
p_t <- (1 - t) * p + t * point
# Multiply the unchanged outcome support by the new probabilities, then sum the two products to obtain the new mean mu_t.
mu_t <- sum(p_t * support_y)
# Subtract the original mean mu from the new mean mu_t. The result delta is one probability difference.
delta <- mu_t - mu
# Subtract the fixed original mean from the selected outcome to obtain the path derivative for this chapter’s mean target.
derivative <- z - mu
# if selects only one branch. At t=0, return NA_real_ instead of computing 0/0; otherwise return the rate of change.
rate <- if (t == 0) NA_real_ else delta / t
# Multiplication precedes addition when computing the new mean. For the mean target, this is an exact linear identity.
linear_mean <- mu + t * derivative
# Subtract the fixed scalar mu from each of the two possible outcomes. The two-element result tabulates D(y)=y−mu under the fixed reference distribution P.
support_D <- support_y - mu
# Apply the same subtraction to all n observed outcomes, returning a numeric vector of patient influence values. Its length is n and its dim is NULL.
influence <- y - risk
# sum adds all n entries; division by n gives one mean. Small floating-point rounding errors are allowed.
mean_influence <- sum(influence) / n
# Weight the two tabulated influence values by their original outcome probabilities and sum. This gives the same mean as averaging the n patient influence values.
grouped_mean_D <- sum(p * support_D)
# Fix the old D and just replace the distribution weights; this time the average is usually non-zero.
old_mean_D <- sum(p_t * support_D)
# Subtract the new mean mu_t from both possible outcomes to tabulate a different influence function in a two-element vector.
new_support_D <- support_y - mu_t
# The new function’s weighted mean under the new distribution is again zero. This differs from the previous line, which uses the old function.
new_mean_D <- sum(p_t * new_support_D)
# Subtract each death indicator from 1 to obtain n survival indicators in the same patient order.
alive <- 1 - y
# Sum the survival indicators and divide by the full cohort size to obtain one 30-day survival probability.
survival <- sum(alive) / n
# Subtract the survival mean from each patient’s survival indicator, giving n values. These are the negatives of the corresponding mortality-mean influence values.
survival_influence <- alive - survival
# Reread the original CSV and summarise it by (x,a,y) cell and (a,x) stratum. Return a list describing the fixed empirical reference distribution.
data <- rhc_data(DEFAULT_SOURCE)
# mean_if uses empirical cell probabilities to calculate the outcome mean and the mean influence value. Return both quantities in a list for comparison with the patient-level calculation.
check <- mean_if(data)
# Check the official cohort and event counts and independent exact-fraction benchmarks for the weighted means of the two outcome categories.
stopifnot(n == 5735L, events == 1918, abs(mu - 1918 / 5735) < 1e-14)
# Check the n patient influence values, two outcome-category probabilities and valid probability sum. Patient rows and outcome support points are different alignments.
stopifnot(length(influence) == n, length(p_t) == 2L, abs(sum(p_t) - 1) < 1e-14, all(p_t >= 0))
# || is short-circuit OR: when t=0 yields NA, skip the undefined rate-of-change check; otherwise check exact linearity and the rate of change.
stopifnot(abs(mu_t - linear_mean) < 1e-14, is.na(rate) || abs(rate - derivative) < 1e-12)
# Require all three correctly centred means to be zero within numerical tolerance. Positive and negative influence values may cancel; individual values need not be zero.
stopifnot(abs(mean_influence) < 1e-12, abs(grouped_mean_D) < 1e-12, abs(new_mean_D) < 1e-12)
# Check that averaging the old function under new weights equals the mean change; that survival IF values are the negatives of mortality IF values; and that grouped and patient-level risks agree.
stopifnot(abs(old_mean_D - delta) < 1e-14, all(abs(survival_influence + influence) < 1e-14), abs(check$risk - risk) < 1e-14)
# cat displays existing objects in order. Semicolons in text separate displayed values; \n starts a new line. No new statistic is calculated.
cat("n =", n, "; events =", events, "; risk =", risk, "; mu_t =", mu_t, "\n")
# Display three distinct quantities separately: the patient-level mean, the old function averaged under new weights, and the new function averaged under new weights.
cat("mean_influence =", mean_influence, "; old_mean_D =", old_mean_D, "; new_mean_D =", new_mean_D, "\n")
# This chapter describes the mean in the same public cohort, without estimating an RHC treatment effect; unknown true P0 differs from empirical Pn.
```

<details><summary>Line-by-line explanation: input, output, dimensions and mathematical expressions</summary>

<div class="r-line-card" data-r-line="4">

#### R line 4

```r
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
```

<p>Input: constants or the current R environment. Output: <code>script_argument</code> · character vector · path or parameter string.</p>

commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.

</div>

<div class="r-line-card" data-r-line="6">

#### R line 6

```r
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
```

<p>Input: constants or the current R environment. Output: <code>source_files</code> · character vector · path or parameter string.</p>

sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.

</div>

<div class="r-line-card" data-r-line="8">

#### R line 8

```r
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch01.R")
```

<p>Input: <code>source_files</code> · character vector · path or parameter string; <code>script_argument</code> · character vector · path or parameter string. Output: <code>script_path</code> · character vector · path or parameter string.</p>

if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.

</div>

<div class="r-line-card" data-r-line="10">

#### R line 10

```r
stopifnot(length(script_path) == 1L, !is.na(script_path))
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.

</div>

<div class="r-line-card" data-r-line="12">

#### R line 12

```r
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: <code>project_root</code> · character vector · path or parameter string.</p>

dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.

</div>

<div class="r-line-card" data-r-line="14">

#### R line 14

```r
source(file.path(project_root, "shared", "R", "RHCCase.R"))
```

<p>Input: <code>project_root</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.

</div>

<div class="r-line-card" data-r-line="16">

#### R line 16

```r
rows <- read_rhc_rows(DEFAULT_SOURCE)
```

<p>Input: constants or the current R environment. Output: <code>rows</code> · data.frame · 5735 rows × 63 columns.</p>

read_rhc_rows reads the official original CSV. The shared file locates DEFAULT_SOURCE. rows is a 5735×63 data.frame; each row represents the same original patient.

</div>

<div class="r-line-card" data-r-line="18">

#### R line 18

```r
observations <- rhc_observations(rows)
```

<p>Input: <code>rows</code> · data.frame · 5735 rows × 63 columns. Output: <code>observations</code> · data.frame · 5735 rows × 3 columns (x, a, y).</p>

Encode the original sex, swang1 and dth30 fields as the binary columns x, a and y. Return an n×3 data.frame in the original row order.

</div>

<div class="r-line-card" data-r-line="20">

#### R line 20

```r
y <- observations$y
```

<p>Input: <code>observations</code> · data.frame · 5735 rows × 3 columns (x, a, y). Output: <code>y</code> · numeric vector · Length n; dim is NULL.</p>

$ selects column y by name. This numeric vector has length n: 1 means death within 30 days, and 0 means survival at 30 days. Its dim is NULL.

$$
\mathbf y=(Y_1,\ldots,Y_n)
$$


</div>

<div class="r-line-card" data-r-line="22">

#### R line 22

```r
n <- length(y)
```

<p>Input: <code>y</code> · numeric vector · Length n; dim is NULL. Output: <code>n</code> · integer scalar · Length 1.</p>

length counts vector elements, returning the integer cohort size n=5735. For a data.frame, nrow counts patient rows while length counts variable columns.

$$
n=5735
$$


</div>

<div class="r-line-card" data-r-line="24">

#### R line 24

```r
events <- sum(y)
```

<p>Input: <code>y</code> · numeric vector · Length n; dim is NULL. Output: <code>events</code> · numeric scalar (double) · Length 1.</p>

sum adds the n binary indicators, returning the death-count scalar 1918. &lt;- assigns the result on the right to the name on the left.

$$
N_Y=\sum_iY_i
$$


</div>

<div class="r-line-card" data-r-line="26">

#### R line 26

```r
risk <- events / n
```

<p>Input: <code>events</code> · numeric scalar (double) · Length 1; <code>n</code> · integer scalar · Length 1. Output: <code>risk</code> · numeric scalar (double) · Length 1.</p>

/ divides the death count by the cohort size. Dividing these two scalars gives one sample 30-day mortality risk on the probability scale.

$$
\widehat\mu=\frac{\sum_iY_i}{n}
$$


</div>

<div class="r-line-card" data-r-line="28">

#### R line 28

```r
support_y <- c(0, 1)
```

<p>Input: constants or the current R environment. Output: <code>support_y</code> · numeric vector · Length 2; aligned by outcomes 0 and 1.</p>

c combines 0 and 1 into a two-element numeric vector. These entries are the two possible outcomes, rather than individual patient records.

$$
\mathcal Y=\{0,1\}
$$


</div>

<div class="r-line-card" data-r-line="30">

#### R line 30

```r
p <- vapply(support_y, function(v) mean(y == v), numeric(1))
```

<p>Input: <code>support_y</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>y</code> · numeric vector · Length n; dim is NULL. Output: <code>p</code> · numeric vector · Length 2; aligned by outcomes 0 and 1.</p>

function(v) defines a rule for one outcome category. == compares each patient’s outcome with v; mean treats TRUE as 1 and FALSE as 0. vapply applies this rule to both categories and requires one number per result, giving p length two.

$$
p_y=n^{-1}\sum_i\mathbf1(Y_i=y)
$$


</div>

<div class="r-line-card" data-r-line="32">

#### R line 32

```r
mu <- sum(p * support_y)
```

<p>Input: <code>p</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>support_y</code> · numeric vector · Length 2; aligned by outcomes 0 and 1. Output: <code>mu</code> · numeric scalar (double) · Length 1.</p>

* multiplies each outcome support value by its probability; sum reduces the two products to one mean. The fixed reference is the empirical outcome distribution.

$$
\mu=\sum_{y=0}^1p_yy=1918/5735
$$


</div>

<div class="r-line-card" data-r-line="34">

#### R line 34

```r
z <- 1
```

<p>Input: constants or the current R environment. Output: <code>z</code> · numeric scalar (double) · Length 1.</p>

Choose outcome 1 to receive the added probability mass. z is one numeric scalar and must be either 0 or 1.

$$
z\in\{0,1\}
$$


</div>

<div class="r-line-card" data-r-line="36">

#### R line 36

```r
t <- 0.1
```

<p>Input: constants or the current R environment. Output: <code>t</code> · numeric scalar (double) · Length 1.</p>

Set the dimensionless replacement proportion to 0.1. With t between 0 and 1, the calculation changes probability weights rather than patients’ recorded y values.

$$
0\leq t\leq1
$$


</div>

<div class="r-line-card" data-r-line="38">

#### R line 38

```r
stopifnot(z %in% support_y, is.finite(t), t >= 0, t <= 1)
```

<p>Input: <code>z</code> · numeric scalar (double) · Length 1; <code>support_y</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>t</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

%in% checks membership in the allowed outcome categories. is.finite excludes NA and infinity; stopifnot checks that the path step is valid.

</div>

<div class="r-line-card" data-r-line="40">

#### R line 40

```r
point <- as.numeric(support_y == z)
```

<p>Input: <code>support_y</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>z</code> · numeric scalar (double) · Length 1. Output: <code>point</code> · numeric vector · Length 2; aligned by outcomes 0 and 1.</p>

== compares both categories with the selected outcome; as.numeric converts FALSE/TRUE to 0/1, giving a two-element point-mass probability vector.

$$
\delta_z(y)=\mathbf1(y=z)
$$


</div>

<div class="r-line-card" data-r-line="42">

#### R line 42

```r
p_t <- (1 - t) * p + t * point
```

<p>Input: <code>t</code> · numeric scalar (double) · Length 1; <code>p</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>point</code> · numeric vector · Length 2; aligned by outcomes 0 and 1. Output: <code>p_t</code> · numeric vector · Length 2; aligned by outcomes 0 and 1.</p>

Evaluate the parentheses first. A scalar scales the vector element by element; adding the two components gives a new two-element probability vector.

$$
p_{t,y}=(1-t)p_y+t\mathbf1(y=z)
$$


</div>

<div class="r-line-card" data-r-line="44">

#### R line 44

```r
mu_t <- sum(p_t * support_y)
```

<p>Input: <code>p_t</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>support_y</code> · numeric vector · Length 2; aligned by outcomes 0 and 1. Output: <code>mu_t</code> · numeric scalar (double) · Length 1.</p>

Multiply the unchanged outcome support by the new probabilities, then sum the two products to obtain the new mean mu_t.

$$
\mu_t=\sum_yp_{t,y}y
$$


</div>

<div class="r-line-card" data-r-line="46">

#### R line 46

```r
delta <- mu_t - mu
```

<p>Input: <code>mu_t</code> · numeric scalar (double) · Length 1; <code>mu</code> · numeric scalar (double) · Length 1. Output: <code>delta</code> · numeric scalar (double) · Length 1.</p>

Subtract the original mean mu from the new mean mu_t. The result delta is one probability difference.

$$
\Delta\mu=\mu_t-\mu
$$


</div>

<div class="r-line-card" data-r-line="48">

#### R line 48

```r
derivative <- z - mu
```

<p>Input: <code>z</code> · numeric scalar (double) · Length 1; <code>mu</code> · numeric scalar (double) · Length 1. Output: <code>derivative</code> · numeric scalar (double) · Length 1.</p>

Subtract the fixed original mean from the selected outcome to obtain the path derivative for this chapter’s mean target.

$$
\dot\mu=z-\mu
$$


</div>

<div class="r-line-card" data-r-line="50">

#### R line 50

```r
rate <- if (t == 0) NA_real_ else delta / t
```

<p>Input: <code>t</code> · numeric scalar (double) · Length 1; <code>delta</code> · numeric scalar (double) · Length 1. Output: <code>rate</code> · numeric scalar (double) · Length 1.</p>

if selects only one branch. At t=0, return NA_real_ instead of computing 0/0; otherwise return the rate of change.

$$
(\mu_t-\mu)/t=z-\mu\quad(t\ne0)
$$


</div>

<div class="r-line-card" data-r-line="52">

#### R line 52

```r
linear_mean <- mu + t * derivative
```

<p>Input: <code>mu</code> · numeric scalar (double) · Length 1; <code>t</code> · numeric scalar (double) · Length 1; <code>derivative</code> · numeric scalar (double) · Length 1. Output: <code>linear_mean</code> · numeric scalar (double) · Length 1.</p>

Multiplication precedes addition when computing the new mean. For the mean target, this is an exact linear identity.

$$
\mu_t=\mu+t(z-\mu)
$$


</div>

<div class="r-line-card" data-r-line="54">

#### R line 54

```r
support_D <- support_y - mu
```

<p>Input: <code>support_y</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>mu</code> · numeric scalar (double) · Length 1. Output: <code>support_D</code> · numeric vector · Length 2; aligned by outcomes 0 and 1.</p>

Subtract the fixed scalar mu from each of the two possible outcomes. The two-element result tabulates D(y)=y−mu under the fixed reference distribution P.

$$
D_P^{\rm mean}(y)=y-\mu
$$


</div>

<div class="r-line-card" data-r-line="56">

#### R line 56

```r
influence <- y - risk
```

<p>Input: <code>y</code> · numeric vector · Length n; dim is NULL; <code>risk</code> · numeric scalar (double) · Length 1. Output: <code>influence</code> · numeric vector · Length n; dim is NULL.</p>

Apply the same subtraction to all n observed outcomes, returning a numeric vector of patient influence values. Its length is n and its dim is NULL.

$$
D_i=Y_i-\widehat\mu
$$


</div>

<div class="r-line-card" data-r-line="58">

#### R line 58

```r
mean_influence <- sum(influence) / n
```

<p>Input: <code>influence</code> · numeric vector · Length n; dim is NULL; <code>n</code> · integer scalar · Length 1. Output: <code>mean_influence</code> · numeric scalar (double) · Length 1.</p>

sum adds all n entries; division by n gives one mean. Small floating-point rounding errors are allowed.

$$
n^{-1}\sum_i(Y_i-\widehat\mu)=0
$$


</div>

<div class="r-line-card" data-r-line="60">

#### R line 60

```r
grouped_mean_D <- sum(p * support_D)
```

<p>Input: <code>p</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>support_D</code> · numeric vector · Length 2; aligned by outcomes 0 and 1. Output: <code>grouped_mean_D</code> · numeric scalar (double) · Length 1.</p>

Weight the two tabulated influence values by their original outcome probabilities and sum. This gives the same mean as averaging the n patient influence values.

$$
\sum_yp_yD_P^{\rm mean}(y)=0
$$


</div>

<div class="r-line-card" data-r-line="62">

#### R line 62

```r
old_mean_D <- sum(p_t * support_D)
```

<p>Input: <code>p_t</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>support_D</code> · numeric vector · Length 2; aligned by outcomes 0 and 1. Output: <code>old_mean_D</code> · numeric scalar (double) · Length 1.</p>

Fix the old D and just replace the distribution weights; this time the average is usually non-zero.

$$
\sum_yp_{t,y}D_P^{\rm mean}(y)=\mu_t-\mu
$$


</div>

<div class="r-line-card" data-r-line="64">

#### R line 64

```r
new_support_D <- support_y - mu_t
```

<p>Input: <code>support_y</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>mu_t</code> · numeric scalar (double) · Length 1. Output: <code>new_support_D</code> · numeric vector · Length 2; aligned by outcomes 0 and 1.</p>

Subtract the new mean mu_t from both possible outcomes to tabulate a different influence function in a two-element vector.

$$
D_{P_t}^{\rm mean}(y)=y-\mu_t
$$


</div>

<div class="r-line-card" data-r-line="66">

#### R line 66

```r
new_mean_D <- sum(p_t * new_support_D)
```

<p>Input: <code>p_t</code> · numeric vector · Length 2; aligned by outcomes 0 and 1; <code>new_support_D</code> · numeric vector · Length 2; aligned by outcomes 0 and 1. Output: <code>new_mean_D</code> · numeric scalar (double) · Length 1.</p>

The new function’s weighted mean under the new distribution is again zero. This differs from the previous line, which uses the old function.

$$
\sum_yp_{t,y}D_{P_t}^{\rm mean}(y)=0
$$


</div>

<div class="r-line-card" data-r-line="68">

#### R line 68

```r
alive <- 1 - y
```

<p>Input: <code>y</code> · numeric vector · Length n; dim is NULL. Output: <code>alive</code> · numeric vector · Length n; dim is NULL.</p>

Subtract each death indicator from 1 to obtain n survival indicators in the same patient order.

$$
L_i=1-Y_i
$$


</div>

<div class="r-line-card" data-r-line="70">

#### R line 70

```r
survival <- sum(alive) / n
```

<p>Input: <code>alive</code> · numeric vector · Length n; dim is NULL; <code>n</code> · integer scalar · Length 1. Output: <code>survival</code> · numeric scalar (double) · Length 1.</p>

Sum the survival indicators and divide by the full cohort size to obtain one 30-day survival probability.

$$
\widehat S(30)=1-\widehat\mu
$$


</div>

<div class="r-line-card" data-r-line="72">

#### R line 72

```r
survival_influence <- alive - survival
```

<p>Input: <code>alive</code> · numeric vector · Length n; dim is NULL; <code>survival</code> · numeric scalar (double) · Length 1. Output: <code>survival_influence</code> · numeric vector · Length n; dim is NULL.</p>

Subtract the survival mean from each patient’s survival indicator, giving n values. These are the negatives of the corresponding mortality-mean influence values.

$$
D_i^{\rm survival}=-D_i^{\rm mean}
$$


</div>

<div class="r-line-card" data-r-line="74">

#### R line 74

```r
data <- rhc_data(DEFAULT_SOURCE)
```

<p>Input: constants or the current R environment. Output: <code>data</code> · list · Named container; each member has its own dimensions.</p>

Reread the original CSV and summarise it by (x,a,y) cell and (a,x) stratum. Return a list describing the fixed empirical reference distribution.

</div>

<div class="r-line-card" data-r-line="76">

#### R line 76

```r
check <- mean_if(data)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>check</code> · list · Named container; each member has its own dimensions.</p>

mean_if uses empirical cell probabilities to calculate the outcome mean and the mean influence value. Return both quantities in a list for comparison with the patient-level calculation.

</div>

<div class="r-line-card" data-r-line="78">

#### R line 78

```r
stopifnot(n == 5735L, events == 1918, abs(mu - 1918 / 5735) < 1e-14)
```

<p>Input: <code>n</code> · integer scalar · Length 1; <code>events</code> · numeric scalar (double) · Length 1; <code>mu</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Check the official cohort and event counts and independent exact-fraction benchmarks for the weighted means of the two outcome categories.

</div>

<div class="r-line-card" data-r-line="80">

#### R line 80

```r
stopifnot(length(influence) == n, length(p_t) == 2L, abs(sum(p_t) - 1) < 1e-14, all(p_t >= 0))
```

<p>Input: <code>influence</code> · numeric vector · Length n; dim is NULL; <code>n</code> · integer scalar · Length 1; <code>p_t</code> · numeric vector · Length 2; aligned by outcomes 0 and 1. Output: checks, displayed output or control flow.</p>

Check the n patient influence values, two outcome-category probabilities and valid probability sum. Patient rows and outcome support points are different alignments.

</div>

<div class="r-line-card" data-r-line="82">

#### R line 82

```r
stopifnot(abs(mu_t - linear_mean) < 1e-14, is.na(rate) || abs(rate - derivative) < 1e-12)
```

<p>Input: <code>mu_t</code> · numeric scalar (double) · Length 1; <code>linear_mean</code> · numeric scalar (double) · Length 1; <code>rate</code> · numeric scalar (double) · Length 1; <code>derivative</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

|| is short-circuit OR: when t=0 yields NA, skip the undefined rate-of-change check; otherwise check exact linearity and the rate of change.

</div>

<div class="r-line-card" data-r-line="84">

#### R line 84

```r
stopifnot(abs(mean_influence) < 1e-12, abs(grouped_mean_D) < 1e-12, abs(new_mean_D) < 1e-12)
```

<p>Input: <code>mean_influence</code> · numeric scalar (double) · Length 1; <code>grouped_mean_D</code> · numeric scalar (double) · Length 1; <code>new_mean_D</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Require all three correctly centred means to be zero within numerical tolerance. Positive and negative influence values may cancel; individual values need not be zero.

</div>

<div class="r-line-card" data-r-line="86">

#### R line 86

```r
stopifnot(abs(old_mean_D - delta) < 1e-14, all(abs(survival_influence + influence) < 1e-14), abs(check$risk - risk) < 1e-14)
```

<p>Input: <code>old_mean_D</code> · numeric scalar (double) · Length 1; <code>delta</code> · numeric scalar (double) · Length 1; <code>survival_influence</code> · numeric vector · Length n; dim is NULL; <code>influence</code> · numeric vector · Length n; dim is NULL; <code>check</code> · list · Named container; each member has its own dimensions; <code>risk</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Check that averaging the old function under new weights equals the mean change; that survival IF values are the negatives of mortality IF values; and that grouped and patient-level risks agree.

</div>

<div class="r-line-card" data-r-line="88">

#### R line 88

```r
cat("n =", n, "; events =", events, "; risk =", risk, "; mu_t =", mu_t, "\n")
```

<p>Input: <code>n</code> · integer scalar · Length 1; <code>events</code> · numeric scalar (double) · Length 1; <code>risk</code> · numeric scalar (double) · Length 1; <code>mu_t</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

cat displays existing objects in order. Semicolons in text separate displayed values; \n starts a new line. No new statistic is calculated.

</div>

<div class="r-line-card" data-r-line="90">

#### R line 90

```r
cat("mean_influence =", mean_influence, "; old_mean_D =", old_mean_D, "; new_mean_D =", new_mean_D, "\n")
```

<p>Input: <code>mean_influence</code> · numeric scalar (double) · Length 1; <code>old_mean_D</code> · numeric scalar (double) · Length 1; <code>new_mean_D</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Display three distinct quantities separately: the patient-level mean, the old function averaged under new weights, and the new function averaged under new weights.

</div>

</details>

</div>

<div class="case-workbench" data-case-study="1"></div>

## Next: describe probability changes with directions and scores

The rule $y-\mu$ used in Chapter 0 now has a derivation: place a small proportion of distributional weight at outcome $y$, and the mean changes by $y-\mu$ per unit replacement proportion. For fixed $P$, the influence values of the two outcomes average to zero using $P$’s probabilities. If another distribution supplies the averaging weights, check the function’s centre again. Linearity of the mean makes these changes exact.

The next chapter returns to the same replacement path and separates two aspects of probability change: how much does a category’s probability change, and how large is that change relative to its original probability? As with the logit path, we will derive its score and pair it with the mean IF, gradually moving towards more general distributional directions.


---

# Chapter 2: Describe probability changes with scores and directional derivatives

Chapter 1 fixed an empirical reference distribution from the same real cohort: death probability $r=1918/5735$ and survival probability $c=3817/5735$. We continue studying mean mortality. The question is: **how do the rates of change in the probabilities combine with $D(y)=y-r$ to give the target’s rate of change?**

## 1. Distinguish step size from the direction of probability change

Continue with replacement towards $z=1$. First subtract old from new in each row:

$$
\begin{aligned}
p_0(t)-p_0&=(1-t)c-c=c-tc-c=-tc,\\
p_1(t)-p_1&=(1-t)r+t-r=r-tr+t-r=t(1-r)=tc.
\end{aligned}
$$

> **Notation recap:** $p=(c,r)$ is the fixed original probability vector and $p(t)$ the new one; $t$ is a dimensionless path parameter. Dividing each row’s change by non-zero $t$ gives direction $h=(-c,c)$. The value $t$ determines how far we move; $h$ determines the allocation of change per unit $t$.

The sum-to-one constraint requires $\sum_jh_j=-c+c=0$: increasing probability in one cell requires removing the same total amount from others. Here a vector is simply a column of numbers ordered by row.

## 2. The score records change relative to the original probability

The same absolute change has a different relative size for common and rare categories. Define the path score:

$$
S_j=\frac{h_j}{p_j},\qquad S=(-c/c,c/r)=(-1,c/r).
$$

> **Notation recap:** $h_j$ is the absolute rate of change in row $j$’s probability, $p_j$ its original probability, and $S_j$ the relative rate. A score describes relative change along a path; the propensity score $g$ describes the probability of receiving RHC given baseline characteristics.

The natural logarithm turns a ratio into a difference: $\log[p_j(t)/p_j]=\log p_j(t)-\log p_j$. The derivative of log at positive $p$ is $1/p$. When the probability itself changes along a path, first calculate how $p$ changes with $t$, then multiply by how log changes with $p$. This is the chain rule, giving:

$$
\left.\frac d{dt}\log p_j(t)\right|_0
=\frac1{p_j}\left.\frac d{dt}p_j(t)\right|_0
=\frac{h_j}{p_j}=S_j.
$$

> **Notation recap:** The derivative is evaluated at the original distribution, $t=0$, so the denominator is the original probability $p_j$. The ratio $h_j/p_j$ gives the relative rate at the starting point; the log difference quotient approaches this as $t\to0$.

The score averages to zero under the original probabilities: $\mathbb E_P S=\sum_jp_j(h_j/p_j)=\sum_jh_j=0$. This is exactly the sum-to-one constraint expressed on a relative scale.

## 3. Why does the gradient–score inner product give the derivative?

For any linear path on finite support, $p_j(t)=p_j+th_j$, calculate the mean’s change directly:

$$
\begin{aligned}
\left.\frac d{dt}\sum_jy_j(p_j+th_j)\right|_0&=\sum_jy_jh_j,\\
\sum_jp_jD_jS_j&=\sum_jp_j(y_j-\mu)\frac{h_j}{p_j}\\
&=\sum_j(y_j-\mu)h_j\\
&=\sum_jy_jh_j-\mu\sum_jh_j\\
&=\sum_jy_jh_j-\mu\times0.
\end{aligned}
$$

> **Notation recap:** $D_j=y_j-\mu$ is the mean IF, and $S$ the current path score. First cancel positive $p_j$; finally use the fact that an admissible direction sums to zero. $\mathbb E_P[DS]$ is an inner product: multiply $D_j$ and $S_j$ in each row, then sum with $P$’s weights.

For point replacement towards $z=1$, the pairing is $c(-r)(-1)+r(c)(c/r)=cr+c^2=c(r+c)=c$, exactly the rate found in Chapter 1. The direction towards 0 has its own $S$; the rule $D$ remains unchanged.

## 4. Include sex and treatment in the observation

Now retain $O=(X,A,Y)$ for each person: $X$ is the original sex code, $A$ early RHC, and $Y$ death. All are 0 or 1, giving $2\times2\times2=8$ possible observation combinations. Counting each combination gives a joint frequency table, with one $(X,A,Y)$ combination per row. Fix a combination $o_*$, with original empirical probability $p_*=n_*/5735$. Multiply every combination’s probability by $1-t$, then add $t$ to the selected one. Thus $S_j=-1+\mathbf1\{j=*\}/p_*$.

The IF for overall mean mortality still uses only $y$. Hence:

$$
\sum_jp_jD_jS_j=-\sum_jp_jD_j+\frac{p_*D_*}{p_*}=0+D_*=y_*-r.
$$

> **Notation recap:** The asterisk marks one selected $(X,A,Y)$ observation type, and $p_*$ is its probability. First use the mean-zero result from Chapter 1 to remove the first term, then cancel $p_*$. We have divided the observations into more detailed categories; the mean target has not changed.

## Does changing step size change the starting-point score?

If $t$ changes from 0.1 to 0.3, does the starting-point score triple? First use the two-category experiment, then select an observation combination in the same joint frequency table to check.

<div class="lab" data-lab="2"></div>
<div class="case-workbench" data-case-study="2"></div>

<details><summary>Answer and explanation</summary>

No. The probability change is $th$, so the change triples; $h$ and $S$ are defined at the original distribution and remain fixed. The denominator of $S$ always uses the original probability to retain the same starting-point direction.

</details>

<details><summary>When can a point mass perturb a distribution?</summary>

All observation combinations in this chapter have positive probability. Selecting a point in the support leaves a small two-sided interval in which the new probabilities stay positive. A continuous distribution $P$ usually assigns zero probability to an individual point; adding mass there changes the support or dominating measure and does not automatically define a regular submodel. Chapter 5 checks the gradient identity for all smooth directions admitted by the model. A mean-zero score expresses the sum-to-one constraint; an IF must also give the correct directional derivatives.

</details>

## Calculate directions, scores and weighted pairings in code

<div class="lesson-language-control"><label>Code language in this chapter <select data-lesson-language><option value="julia">Julia operations</option><option value="r">R operations</option></select></label><span>Use the same RHC data and the same mathematical objectives</span></div>

<div class="lesson-code-panel" data-lesson-panel="julia">


The input remains the same original CSV. The script outputs the selected cell’s direction $h$, its score $S$, the mean’s rate of change and the pairing. The web workbench also lists original and new probabilities, $D$, and each cell’s $pDS$. Observe how the final pairing remains the derivative at the reference distribution when $t$ changes.

The input is [the same original RHC CSV](../shared/data/public/rhc/source.csv). For reading and fitting definitions, see [RHCCase.jl](../shared/julia/RHCCase.jl). The complete [chapter script](julia/rhc_lessons/ch02.jl) is editable; all scripts can be run from other working directories, as long as the relative positions of en/ and shared/ are retained.

```sh
julia --startup-file=no --history-file=no en/julia/rhc_lessons/ch02.jl
```

```julia
# Public RHC cohort: X=sex, A=RHC receipt, Y=30-day death; all quantities are computed from original records.
# See BEGIN_CH02 in EIFCase.jl for core definitions; this chapter still studies only the endpoint-event mean.
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
using .RHCCase
using Printf

data = rhc_data()
direction = point_direction(data, 4) # Cell 4: x=0,a=1,y=1; indices start at 1.
p = [c.p for c in data.cells]
t = 0.01
pt = linear_path(p, direction.score, t)
new_mean = sum(pt[i] * c.y for (i,c) in enumerate(data.cells))
@printf("Original risk=%.6f, New risk=%.6f, Rate of change=%.6f\n", data.marginal_risk, new_mean, (new_mean-data.marginal_risk)/t)
@printf("Direct derivative=%.6f, IF×score pairing=%.6f\n", direction.derivative, direction.pairing)
for (i,c) in enumerate(data.cells)
    @printf("(%d,%d,%d): h=%+.6f, score=%+.6f\n", c.x, c.a, c.y, direction.h[i], direction.score[i])
end
```


<details><summary>View the source for this chapter’s mathematical functions</summary>

RHCCase reuses the following functions with counts from the source data. They contain no built-in values for the unknown population target.

```julia
# Chapter 2: t selects the path position; h, score and the derivative at the starting point stay fixed.
function point_direction(data, index::Integer)
    1 <= index <= length(data.cells) || throw(ArgumentError("Invalid cell index"))
    p = [c.p for c in data.cells]
    h = [(i == index ? 1.0 : 0.0) - p[i] for i in eachindex(p)]
    score = h ./ p
    influence = mean_if(data)
    derivative = sum(c.y * h[i] for (i, c) in enumerate(data.cells))
    pairing = weighted_inner(p, influence.values, score)
    (; index, h, score, derivative, pairing)
end
function linear_path(p, score, t::Real)
    checked_probabilities(p); checked_vector(score, length(p))
    isfinite(t) || throw(ArgumentError("Finite path parameter required"))
    abs(dot(p, score)) <= 1e-12 || throw(ArgumentError("Score must be centred under p"))
    updated = p .* (1 .+ t .* score)
    checked_probabilities(updated)  # No truncation: fail outside the current strictly positive probability model.
    updated
end
```

</details>



</div>

<div class="lesson-code-panel" data-lesson-panel="r" hidden>

The input is [the same original public RHC CSV](../shared/data/public/rhc/source.csv). Shared definitions: [RHCCase.R](../shared/R/RHCCase.R) and [EIFCase.R](../shared/R/EIFCase.R). [Complete R script](R/ch02.R) uses base R and runs from any working directory if you retain the project directory structure.

```sh
Rscript --vanilla en/R/ch02.R
```

```r
# Chapter 2: probability directions and mean derivatives
# Input: frozen original public RHC CSV; output: chapter objects and terminal checks; base R only.
# commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
# sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
# if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch02.R")
# length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.
stopifnot(length(script_path) == 1L, !is.na(script_path))
# dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
# source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.
source(file.path(project_root, "shared", "R", "RHCCase.R"))
# Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.
data <- rhc_data()
# The fourth cell is (x,a,y)=(0,1,1); R uses one-based indexing. Return a list containing h, score, derivative and pairing.
direction <- point_direction(data, 4L)
# Two successive $ operators select the cells table from data and then its p column, producing an eight-element probability vector.
p <- data$cells$p
# Set the dimensionless path step size t to one numeric scalar.
t <- 0.01
# Calculate eight new probabilities in the same cell order; the shared function rejects invalid probabilities.
pt <- linear_path(p, direction$score, t)
# Multiply each new cell probability by the corresponding outcome, then sum the eight products to obtain the new outcome mean.
new_mean <- sum(pt * data$cells$y)
# Display the original mean, new mean and difference quotient. Here t is non-zero, so division by t is defined.
cat("risk =", data$marginal_risk, "; new risk =", new_mean, "; rate =", (new_mean - data$marginal_risk) / t, "\n")
# Display the direct derivative and the pairing of the influence function with the score. The two values should agree.
cat("derivative =", direction$derivative, "; IF-score pairing =", direction$pairing, "\n")
# [c(...)] selects three columns. data.frame joins h and score for the same cells; print displays eight rows, preserving the original data.
print(data.frame(data$cells[c("x", "a", "y")], h = direction$h, score = direction$score))
```

<details><summary>Line-by-line explanation: input, output, dimensions and mathematical expressions</summary>

<div class="r-line-card" data-r-line="4">

#### R line 4

```r
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
```

<p>Input: constants or the current R environment. Output: <code>script_argument</code> · character vector · path or parameter string.</p>

commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.

</div>

<div class="r-line-card" data-r-line="6">

#### R line 6

```r
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
```

<p>Input: constants or the current R environment. Output: <code>source_files</code> · character vector · path or parameter string.</p>

sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.

</div>

<div class="r-line-card" data-r-line="8">

#### R line 8

```r
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch02.R")
```

<p>Input: <code>source_files</code> · character vector · path or parameter string; <code>script_argument</code> · character vector · path or parameter string. Output: <code>script_path</code> · character vector · path or parameter string.</p>

if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.

</div>

<div class="r-line-card" data-r-line="10">

#### R line 10

```r
stopifnot(length(script_path) == 1L, !is.na(script_path))
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.

</div>

<div class="r-line-card" data-r-line="12">

#### R line 12

```r
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: <code>project_root</code> · character vector · path or parameter string.</p>

dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.

</div>

<div class="r-line-card" data-r-line="14">

#### R line 14

```r
source(file.path(project_root, "shared", "R", "RHCCase.R"))
```

<p>Input: <code>project_root</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.

</div>

<div class="r-line-card" data-r-line="16">

#### R line 16

```r
data <- rhc_data()
```

<p>Input: constants or the current R environment. Output: <code>data</code> · list · Named container; each member has its own dimensions.</p>

Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.

</div>

<div class="r-line-card" data-r-line="18">

#### R line 18

```r
direction <- point_direction(data, 4L)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>direction</code> · list · Named container; each member has its own dimensions.</p>

The fourth cell is (x,a,y)=(0,1,1); R uses one-based indexing. Return a list containing h, score, derivative and pairing.

$$
h_j=\mathbf1(j=4)-p_j,\quad S_j=h_j/p_j
$$


</div>

<div class="r-line-card" data-r-line="20">

#### R line 20

```r
p <- data$cells$p
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>p</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Two successive $ operators select the cells table from data and then its p column, producing an eight-element probability vector.

</div>

<div class="r-line-card" data-r-line="22">

#### R line 22

```r
t <- 0.01
```

<p>Input: constants or the current R environment. Output: <code>t</code> · numeric scalar (double) · Length 1.</p>

Set the dimensionless path step size t to one numeric scalar.

</div>

<div class="r-line-card" data-r-line="24">

#### R line 24

```r
pt <- linear_path(p, direction$score, t)
```

<p>Input: <code>p</code> · numeric vector · Length 8; aligned according to (x, a, y) grid; <code>direction</code> · list · Named container; each member has its own dimensions; <code>t</code> · numeric scalar (double) · Length 1. Output: <code>pt</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Calculate eight new probabilities in the same cell order; the shared function rejects invalid probabilities.

$$
p_{t,j}=p_j(1+tS_j)
$$


</div>

<div class="r-line-card" data-r-line="26">

#### R line 26

```r
new_mean <- sum(pt * data$cells$y)
```

<p>Input: <code>pt</code> · numeric vector · Length 8; aligned according to (x, a, y) grid; <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>new_mean</code> · numeric scalar (double) · Length 1.</p>

Multiply each new cell probability by the corresponding outcome, then sum the eight products to obtain the new outcome mean.

$$
\mu_t=\sum_jp_{t,j}y_j
$$


</div>

<div class="r-line-card" data-r-line="28">

#### R line 28

```r
cat("risk =", data$marginal_risk, "; new risk =", new_mean, "; rate =", (new_mean - data$marginal_risk) / t, "\n")
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>new_mean</code> · numeric scalar (double) · Length 1; <code>t</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Display the original mean, new mean and difference quotient. Here t is non-zero, so division by t is defined.

</div>

<div class="r-line-card" data-r-line="30">

#### R line 30

```r
cat("derivative =", direction$derivative, "; IF-score pairing =", direction$pairing, "\n")
```

<p>Input: <code>direction</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the direct derivative and the pairing of the influence function with the score. The two values should agree.

$$
\dot\mu=\sum_jp_jD_jS_j
$$


</div>

<div class="r-line-card" data-r-line="32">

#### R line 32

```r
print(data.frame(data$cells[c("x", "a", "y")], h = direction$h, score = direction$score))
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>direction</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

[c(...)] selects three columns. data.frame joins h and score for the same cells; print displays eight rows, preserving the original data.

</div>

</details>

</div>

## Next: factorise the joint distribution into three mechanisms

We have distinguished step size, probability change, direction, score and directional derivative, and verified the pairing for the mean. The next chapter factorises the joint distribution into three mechanisms to establish which ones the standardised target uses.


---

# Chapter 3: Use the joint distribution to define a standardised risk difference

Overall mortality does not distinguish why the RHC and no-RHC groups differ. Starting from real cohort counts grouped by sex, treatment and outcome, this chapter identifies each denominator and constructs a standardised target comparing both treatments in the same population.

## 1. Calculate conditional probabilities within a specified group

The original sex field Female is coded $X=0$, and Male is $X=1$. All counts below come from the original CSV and appear in the workbench. Specifically, there are 2543 women, of whom 906 received RHC; 348 of those 906 died. Thus 348 is the number of **deaths among women receiving RHC**. Different denominators answer different questions:

$$
p_X(0)=\frac{2543}{5735},\quad
g(0)=\frac{906}{2543},\quad
Q(1,0)=\frac{348}{906}.
$$

> **Notation recap:** $p_X$ describes population composition, $g$ the RHC probability given $X$, and $Q$ mortality risk given $A,X$. The conditional mean $Q$ equals the within-group average of 0/1 outcomes. Here we use a saturated empirical fit to the current table: a separate mortality risk for each $(a,x)$ group.

## 2. Why can the joint distribution be factorised?

Multiplying the conditional probabilities cancels the successive denominators:

$$
\begin{aligned}
p(x,a,y)&=p(y\mid a,x)p(a\mid x)p(x),\\
p(0,1,1)&=\frac{348}{906}\frac{906}{2543}\frac{2543}{5735}=\frac{348}{5735}.
\end{aligned}
$$

> **Notation recap:** The first line factorises into the $Y$, $A$ and $X$ mechanisms; the second is the cell for women receiving RHC who died. Although written $p(0,1,1)$, the coordinates remain ordered $X,A,Y$; the multiplication order does not change their order.

<div class="case-workbench" data-case-study="3"></div>

## 3. Why do both arms use the same population weights after standardisation?

Predict risk under each treatment for the same group of people:

$$
\Psi_X(P)=\sum_xp_X(x)\{Q(1,x)-Q(0,x)\}.
$$

> **Notation recap:** Subscript $X$ emphasises that this example adjusts only for sex; both arms use the same population weights $p_X$. The real table gives approximately 0.0736921, or 7.36921 percentage points, a sex-standardised observational association.

The unadjusted difference uses different weights: $\mathbb E[Y\mid A=1]-\mathbb E[Y\mid A=0]$. The first term weights women by 906/2184, the second by 1637/3551. Compare these with the common standardisation weight 2543/5735 to see how the two calculations use population composition.

## 4. What happens if only one mechanism changes?

For fixed $Q$ and $p_X$, the standardisation formula contains no $g$, so changing $g$ alone leaves the target unchanged. **The treatment mechanism $g$ determines how much outcome information each treatment stratum provides, affecting the precision of $Q$ estimation.** For example, if RHC is rare in a stratum, its RHC risk is harder to estimate. The denominators in Chapter 4 and variance in Chapter 7 explain this.

<div class="lab" data-lab="3"></div>

Predict first: if both strata have $g=0.5$, what happens to the standardised and unadjusted differences? Then change one $Q$ and observe both.

<details><summary>Answer and explanation</summary>

With both $g$ values equal to 0.5, the observed arms have the same $X$ composition, given by the original $p_X$. The unadjusted difference therefore equals the standardised difference, which has not changed. Changing $Q$ changes one group’s risk and usually changes both differences. The interface normalises within each conditional group separately; a pure $g$ operation always holds $Q$ and $p_X$ fixed.

</details>

## 5. Fit initial risks using a treatment-only model

Start with a risk model containing only $A$: one mortality proportion per arm, shared across $X$. It deliberately ignores within-arm differences by sex, making the role of targeting easier to see. Estimate the initial $g$ from the observed treatment proportion in each sex group, and $p_X$ from observed counts. The saturated $Q$ and initial fitted $Q$ are different functions; the workbench lists them separately, and subsequent chapters keep them distinct.

## Code workbook: input, operation and output

<div class="lesson-language-control"><label>Code language in this chapter <select data-lesson-language><option value="julia">Julia operations</option><option value="r">R operations</option></select></label><span>Use the same RHC data and the same mathematical objectives</span></div>

<div class="lesson-code-panel" data-lesson-panel="julia">

The input is [the same original RHC CSV](../shared/data/public/rhc/source.csv). For reading and fitting definitions, see [RHCCase.jl](../shared/julia/RHCCase.jl). The complete [chapter script](julia/rhc_lessons/ch03.jl) is editable; all scripts can be run from other working directories, as long as the relative positions of en/ and shared/ are retained.

```sh
julia --startup-file=no --history-file=no en/julia/rhc_lessons/ch03.jl
```

```julia
# Public RHC cohort: X=sex, A=RHC receipt, Y=30-day death; all quantities are computed from original records.
# See BEGIN_CH03 in EIFCase.jl for reading and four-group summary definitions.
# q[a+1,x+1]: rows index treatment a, columns index baseline x; fits use only supplied counts.
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
using .RHCCase
using Printf

data = rhc_data()
for group in data.groups
    @printf("x=%d,a=%d: Patient count=%d,Events=%d,Risk=%.6f,Initial Q=%.6f\n", group.x, group.a, group.n, group.events, group.q_observed, group.initial_q)
end
println("Empirical population proportions px = ", data.empirical_model.px)
println("Treatment probabilities g estimated by X = ", data.empirical_model.g)
@printf("Initial risk difference after fitting on A alone=%.6f\n", data.initial_estimate)

println("Standardisation here uses sex alone, without establishing sufficient confounding control; results are not clinical causal effects of RHC.")
```


<details><summary>View the source for this chapter’s mathematical functions</summary>

RHCCase reuses the following functions with counts from the source data. They contain no built-in values for the unknown population target.

```julia
# Keep x/a/y labels as 0/1; add 1 for array indices. Q rows index a and columns index x.
# This module reads only caller-supplied counts, rather than the JavaScript teaching model.
function read_counts(path::AbstractString)
    lines = readlines(path)
    isempty(lines) && throw(ArgumentError("Empty counts CSV"))
    strip(lines[1]) == "x,a,y,n" || throw(ArgumentError("Expected CSV header x,a,y,n"))
    rows = NamedTuple{(:x, :a, :y, :n), Tuple{Int,Int,Int,Int}}[]
    for line in lines[2:end]
        isempty(strip(line)) && continue
        fields = split(strip(line), ',')
        length(fields) == 4 || throw(ArgumentError("Expected four integer CSV fields"))
        x, a, y, n = parse.(Int, strip.(fields))
        push!(rows, (; x, a, y, n))
    end
    checked_counts(rows)
end

function checked_counts(rows)
    length(rows) == 8 || throw(ArgumentError("The teaching model requires eight positive cells"))
    checked = NamedTuple{(:x, :a, :y, :n), Tuple{Int,Int,Int,Int}}[]
    for c in rows
        all(k -> hasproperty(c, k), (:x, :a, :y, :n)) ||
            throw(ArgumentError("Each cell needs x,a,y,n"))
        all(v -> v isa Integer && !(v isa Bool) && v in (0, 1), (c.x, c.a, c.y)) ||
            throw(ArgumentError("Cell labels must be binary integers"))
        c.n isa Integer && !(c.n isa Bool) && 0 < c.n <= typemax(Int) ||
            throw(ArgumentError("Each observed count must be a positive integer"))
        push!(checked, (x=Int(c.x), a=Int(c.a), y=Int(c.y), n=Int(c.n)))
    end
    length(Set((c.x, c.a, c.y) for c in checked)) == 8 ||
        throw(ArgumentError("Duplicate cell labels"))
    sort(checked; by=c -> (c.x, c.a, c.y))
end

function case_data(rows::AbstractVector)
    counts = checked_counts(rows)
    n = foldl((s, c) -> Base.Checked.checked_add(s, c.n), counts; init=0)
    cells = [merge(c, (p=c.n / n,)) for c in counts]
    px = [sum(c.n for c in counts if c.x == x) / n for x in 0:1]
    g = [sum(c.n for c in counts if c.x == x && c.a == 1) /
         sum(c.n for c in counts if c.x == x) for x in 0:1]
    q = [sum(c.n * c.y for c in counts if c.a == a && c.x == x) /
         sum(c.n for c in counts if c.a == a && c.x == x) for a in 0:1, x in 0:1]
    arm_risks = [sum(c.n * c.y for c in counts if c.a == a) /
                 sum(c.n for c in counts if c.a == a) for a in 0:1]
    empirical_model = (px=copy(px), g=copy(g), q=copy(q))
    initial_fit = (px=copy(px), g=copy(g), q=[arm_risks[a+1] for a in 0:1, x in 0:1])
    groups = [(x=x, a=a,
               n=sum(c.n for c in counts if c.x == x && c.a == a),
               events=sum(c.n * c.y for c in counts if c.x == x && c.a == a),
               q_observed=q[a+1,x+1], p_x=px[x+1], g_hat=g[x+1],
               initial_q=initial_fit.q[a+1,x+1]) for x in 0:1 for a in 0:1]
    (; n, cells, groups, empirical_model, initial_fit,
       marginal_risk=sum(c.p * c.y for c in cells),
       initial_estimate=ate(initial_fit), observed_difference=arm_risks[2] - arm_risks[1])
end
case_data(path::AbstractString) = case_data(read_counts(path))

ate(model) = sum(model.px[x+1] * (model.q[2,x+1] - model.q[1,x+1]) for x in 0:1)
```

</details>

</div>

<div class="lesson-code-panel" data-lesson-panel="r" hidden>

The input is [the same original public RHC CSV](../shared/data/public/rhc/source.csv). Shared definitions: [RHCCase.R](../shared/R/RHCCase.R) and [EIFCase.R](../shared/R/EIFCase.R). [Complete R script](R/ch03.R) uses base R and runs from any working directory if you retain the project directory structure.

```sh
Rscript --vanilla en/R/ch03.R
```

```r
# Chapter 3: empirical objects for three mechanisms
# Input: frozen original public RHC CSV; output: chapter objects and terminal checks; base R only.
# commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
# sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
# if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch03.R")
# length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.
stopifnot(length(script_path) == 1L, !is.na(script_path))
# dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
# source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.
source(file.path(project_root, "shared", "R", "RHCCase.R"))
# Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.
data <- rhc_data()
# Display a multi-column table with four rows, one per (x,a) group. n, events, q_observed and initial_q are aggregated from the original CSV.
print(data$groups)
# Extract the two-element vector of empirical X probabilities, ordered as x=0 then x=1.
px <- data$empirical_model$px
# Extract the two-element vector of conditional RHC probabilities. Entry x+1 corresponds to baseline stratum x.
g <- data$empirical_model$g
# Extract the 2×2 conditional death risk matrix: row a+1, column x+1; q[1,2] is a=0, x=1.
q <- data$empirical_model$q
# Display empirical population proportions, treatment probabilities, and the standardised risk difference from the initial Q fitted using A alone.
cat("px =", px, "; g =", g, "; initial risk difference =", data$initial_estimate, "\n")
# print preserves the 2×2 risk matrix’s rows and columns. Matrix rows represent treatment arms, rather than patients.
print(q)
# Standardisation uses sex alone; sufficient confounding control is unproven, so results are not identified clinical causal effects.
```

<details><summary>Line-by-line explanation: input, output, dimensions and mathematical expressions</summary>

<div class="r-line-card" data-r-line="4">

#### R line 4

```r
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
```

<p>Input: constants or the current R environment. Output: <code>script_argument</code> · character vector · path or parameter string.</p>

commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.

</div>

<div class="r-line-card" data-r-line="6">

#### R line 6

```r
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
```

<p>Input: constants or the current R environment. Output: <code>source_files</code> · character vector · path or parameter string.</p>

sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.

</div>

<div class="r-line-card" data-r-line="8">

#### R line 8

```r
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch03.R")
```

<p>Input: <code>source_files</code> · character vector · path or parameter string; <code>script_argument</code> · character vector · path or parameter string. Output: <code>script_path</code> · character vector · path or parameter string.</p>

if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.

</div>

<div class="r-line-card" data-r-line="10">

#### R line 10

```r
stopifnot(length(script_path) == 1L, !is.na(script_path))
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.

</div>

<div class="r-line-card" data-r-line="12">

#### R line 12

```r
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: <code>project_root</code> · character vector · path or parameter string.</p>

dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.

</div>

<div class="r-line-card" data-r-line="14">

#### R line 14

```r
source(file.path(project_root, "shared", "R", "RHCCase.R"))
```

<p>Input: <code>project_root</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.

</div>

<div class="r-line-card" data-r-line="16">

#### R line 16

```r
data <- rhc_data()
```

<p>Input: constants or the current R environment. Output: <code>data</code> · list · Named container; each member has its own dimensions.</p>

Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.

</div>

<div class="r-line-card" data-r-line="18">

#### R line 18

```r
print(data$groups)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display a multi-column table with four rows, one per (x,a) group. n, events, q_observed and initial_q are aggregated from the original CSV.

</div>

<div class="r-line-card" data-r-line="20">

#### R line 20

```r
px <- data$empirical_model$px
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>px</code> · numeric vector · Length 2; aligned by x=0,1.</p>

Extract the two-element vector of empirical X probabilities, ordered as x=0 then x=1.

$$
p_x=P_n(X=x)
$$


</div>

<div class="r-line-card" data-r-line="22">

#### R line 22

```r
g <- data$empirical_model$g
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>g</code> · numeric vector · Length 2; aligned by x=0,1.</p>

Extract the two-element vector of conditional RHC probabilities. Entry x+1 corresponds to baseline stratum x.

$$
g(x)=P_n(A=1\mid X=x)
$$


</div>

<div class="r-line-card" data-r-line="24">

#### R line 24

```r
q <- data$empirical_model$q
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>q</code> · numeric matrix · 2 rows (a+1) × 2 columns (x+1).</p>

Extract the 2×2 conditional death risk matrix: row a+1, column x+1; q[1,2] is a=0, x=1.

$$
Q(a,x)=E_{P_n}(Y\mid A=a,X=x)
$$


</div>

<div class="r-line-card" data-r-line="26">

#### R line 26

```r
cat("px =", px, "; g =", g, "; initial risk difference =", data$initial_estimate, "\n")
```

<p>Input: <code>px</code> · numeric vector · Length 2; aligned by x=0,1; <code>g</code> · numeric vector · Length 2; aligned by x=0,1; <code>data</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display empirical population proportions, treatment probabilities, and the standardised risk difference from the initial Q fitted using A alone.

</div>

<div class="r-line-card" data-r-line="28">

#### R line 28

```r
print(q)
```

<p>Input: <code>q</code> · numeric matrix · 2 rows (a+1) × 2 columns (x+1). Output: checks, displayed output or control flow.</p>

print preserves the 2×2 risk matrix’s rows and columns. Matrix rows represent treatment arms, rather than patients.

</div>

</details>

</div>



<details><summary>How does adding baseline variables change the standardised target?</summary>

Using sex here gives a sex-standardised association. When Chapter 8 expands to $W$, the outer average uses all patients’ baseline vectors; $Q$ and $g$ still denote conditional mortality risk and treatment probability. The adjustment set determines the statistical functional. A causal interpretation additionally requires justification of Chapter 0’s confounding-control and selection conditions. This course uses confirmed, complete 30-day outcomes.

</details>

## Next: derive the gradient of the standardised risk difference

We have factorised the distribution into three mechanisms, defined the standardised target and fitted an initial $Q$ using the same cohort. The next chapter seeks a function representing the target’s rate of change, starting with differentiation of the conditional mean’s numerator and denominator to construct the EIF term by term.


---

# Chapter 4: Derive the EIF by differentiating conditional means

Grouped frequencies from the same RHC cohort have supplied an empirical reference distribution. This chapter asks: **if we increase probability at one observation combination, how does the standardised risk difference change?** We will find inverse probability weights in the changing denominators.

## 1. Write conditional risk as a ratio

Fix treatment arm $a$ and baseline stratum $x$. Let $b=\mathbb E_P[Y\mathbf1\{A=a,X=x\}]$ and $v=P(A=a,X=x)>0$, so $Q(a,x)=b/v$. For point replacement towards observation $o_*=(x_*,a_*,y_*)$, if it belongs to this group:

$$
b_t=(1-t)b+ty_*,\qquad v_t=(1-t)v+t.
$$

> **Notation recap:** $b$ is this group’s contribution to the population event probability, and $v$ is the group’s population probability; both are measured on the probability scale. The value $t$ is the probability path parameter, and $y_*$ the outcome code of the selected observation type.

## 2. Subtract with a common denominator before differentiating

Start without memorising the quotient rule:

$$
\begin{aligned}
Q_t-Q&=\frac{b+t(y_*-b)}{v+t(1-v)}-\frac bv\\
&=\frac{v[b+t(y_*-b)]-b[v+t(1-v)]}{v[v+t(1-v)]}\\
&=\frac{vb+vt(y_*-b)-bv-bt(1-v)}{v[v+t(1-v)]}\\
&=\frac{t[vy_*-vb-b+bv]}{v[v+t(1-v)]}\\
&=\frac{t(vy_*-b)}{v[v+t(1-v)]}\\
&=\frac{t(y_*-Q)}{v+t(1-v)}.
\end{aligned}
$$

> **Notation recap:** $Q=b/v$ is the fixed original risk; $vb-bv$ and $-vb+bv$ cancel in succession. Finally cancel $v$ between numerator and denominator. Divide by $t$, then let $t$ approach 0, obtaining $(y_*-Q)/v$. Positive $v$ permits cancellation and ensures that the derivative exists near the reference distribution.

If the selected point is outside this group, numerator and denominator both multiply by $1-t$, leaving the ratio exactly unchanged. Therefore:

$$
\dot Q(a,x)=\frac{\mathbf1\{a_*=a,x_*=x\}}{p_X(x)\pi_a(x)}[y_*-Q(a,x)],
\quad\pi_1=g,\ \pi_0=1-g.
$$

> **Notation recap:** $\pi_a$ is the conditional probability of actually receiving $a$; residual $y_*-Q$ arises from the numerator change minus the denominator change in the quotient.

## 3. The outer average also changes

Write $\tau(x)=Q(1,x)-Q(0,x)$ and target $\psi=\sum_xp_X(x)\tau(x)$. Begin with the exact difference: $p_t\tau_t-p\tau=(p_t-p)\tau+p(\tau_t-\tau)+(p_t-p)(\tau_t-\tau)$. After dividing by $t$, the first two terms approach $\dot p\tau$ and $p\dot\tau$. The last term is $[(p_t-p)/t](\tau_t-\tau)$: its first factor has a finite limit and its second approaches zero, so the product approaches zero. We are therefore taking the first-order derivative at the starting point.

Along point replacement, the outer weights change at rate $\dot p_X(x)=\mathbf1\{x=x_*\}-p_X(x)$, giving outer term $\tau(x_*)-\psi$. The inner $p_X(x)$ cancels the corresponding denominator in the previous section, leaving only the reciprocal of $\pi_a$. The control risk is subtracted, so the residual term is:

$$
\frac{a_*}{g(x_*)}[y_*-Q(1,x_*)]
-\frac{1-a_*}{1-g(x_*)}[y_*-Q(0,x_*)].
$$

> **Notation recap:** $a_*$ is either 0 or 1, acting as a switch selecting the observed treatment arm. The minus sign comes from the target definition $Q_1-Q_0$; the term’s final sign also depends on the residual.

## 4. Combine the residual and centring terms and check the candidate gradient’s mean

Combine these two terms into a function of the observation $o=(x,a,y)$:

$$
\begin{aligned}
D_P(o)&=H_g(a,x)[y-Q(a,x)]+\tau(x)-\psi,\\
H_g(a,x)&=\frac a{g(x)}-\frac{1-a}{1-g(x)}.
\end{aligned}
$$

> **Notation recap:** $H$ is the signed inverse probability weight for the observed treatment. $D$ is a candidate gradient for this target, whose values give the change in risk difference per unit path step. Subscript $P$ indicates that $Q,g,\psi$ all come from the same reference distribution.

Given $A,X$, the residual has conditional expectation zero, so the weighted residual also averages to zero. The outer term has mean $\mathbb E\tau-\psi=0$. This establishes centring, but mean zero alone does not prove correct derivatives in every direction; Chapter 5 continues the verification.

## How does switching treatment arms change the residual term’s sign?

Choose a cell with $Y=1$ but $A=0$: must the residual term be positive? If we remove the inverse probability denominator, will it still agree with direct differencing?

<div class="lab" data-lab="4"></div>
<div class="case-workbench" data-case-study="4"></div>

<details><summary>Answer and explanation</summary>

When $y=1$ and $Q<1$, the residual is positive but $H$ for $A=0$ is negative, so the outcome term is negative. The full $D$ also includes the outer term, so calculate both terms to determine its sign. Removing the denominator generally fails because the denominator of the within-group mean contains $v$; the outer weight cancels only $p_X$, leaving the treatment probability in the denominator. The webpage recalculates the target from perturbed observation-combination probabilities, independently of the formula for $D$.

</details>

## Calculate initial-fit weighted residuals in code

<div class="lesson-language-control"><label>Code language in this chapter <select data-lesson-language><option value="julia">Julia operations</option><option value="r">R operations</option></select></label><span>Use the same RHC data and the same mathematical objectives</span></div>

<div class="lesson-code-panel" data-lesson-panel="julia">


The initial $Q$ is fitted using only $A$, unlike the saturated $Q$ used for this chapter’s mathematical derivatives. The script applies the same $H$ to initial-model residuals, outputting $H$ and the average initial score. The web workbench also lists contributions by treatment–sex group and fitted influence-function values for each observation combination. This sample average is generally non-zero, the quantity targeted in Chapter 8; it measures the weighted residual left by initial predictions in these data.

The input is [the same original RHC CSV](../shared/data/public/rhc/source.csv). For reading and fitting definitions, see [RHCCase.jl](../shared/julia/RHCCase.jl). The complete [chapter script](julia/rhc_lessons/ch04.jl) is editable; all scripts can be run from other working directories, as long as the relative positions of en/ and shared/ are retained.

```sh
julia --startup-file=no --history-file=no en/julia/rhc_lessons/ch04.jl
```

```julia
# Public RHC cohort: X=sex, A=RHC receipt, Y=30-day death; all quantities are computed from original records.
# See BEGIN_CH04 in EIFCase.jl for H and gradient definitions; Q is not updated here yet.
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
using .RHCCase
using Printf

data = rhc_data()
initial_d = eif_values(data.initial_fit, data.cells)
for group in data.groups
    h = clever_h(data.initial_fit, group.a, group.x)
    @printf("x=%d,a=%d: H=%+.6f\n", group.x, group.a, h)
end
correction = sum(c.p * initial_d[i] for (i,c) in enumerate(data.cells))
@printf("Initial target=%+.6f, Sample mean of the initially fitted gradient=%+.6f\n", data.initial_estimate, correction)
println("This chapter retains the initial target and empirical gradient mean; Chapter 8 constructs the correction and Q update.")

println("Standardisation here uses sex alone, without establishing sufficient confounding control; results are not clinical causal effects of RHC.")
```


<details><summary>View the source for this chapter’s mathematical functions</summary>

RHCCase reuses the following functions with counts from the source data. They contain no built-in values for the unknown population target.

```julia
function clever_h(model, a, x)
    a in (0, 1) && x in (0, 1) || throw(ArgumentError("Binary group labels required"))
    g = model.g[x+1]
    0 < g < 1 || throw(ArgumentError("Treatment probabilities must be strictly interior"))
    a == 1 ? 1 / g : -1 / (1 - g)
end

function eif_values(model, cells; estimate=ate(model))
    [model.q[2,c.x+1] - model.q[1,c.x+1] - estimate +
     clever_h(model, c.a, c.x) * (c.y - model.q[c.a+1,c.x+1]) for c in cells]
end
```

</details>

<details><summary>Does the derivation still hold with multiple baseline variables?</summary>

The conditional-mean quotient, treatment switch and outer centring do not require binary $X$. For general $W$, conditional expectations and integrals express the same decomposition. However, point replacement with continuous $W$ does not automatically define a regular submodel. Checking all admissible smooth score directions is therefore the mathematical task for the next chapter.

</details>



</div>

<div class="lesson-code-panel" data-lesson-panel="r" hidden>

The input is [the same original public RHC CSV](../shared/data/public/rhc/source.csv). Shared definitions: [RHCCase.R](../shared/R/RHCCase.R) and [EIFCase.R](../shared/R/EIFCase.R). [Complete R script](R/ch04.R) uses base R and runs from any working directory if you retain the project directory structure.

```sh
Rscript --vanilla en/R/ch04.R
```

```r
# Chapter 4: cell-by-cell EIF components
# Input: frozen original public RHC CSV; output: chapter objects and terminal checks; base R only.
# commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
# sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
# if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch04.R")
# length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.
stopifnot(length(script_path) == 1L, !is.na(script_path))
# dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
# source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.
source(file.path(project_root, "shared", "R", "RHCCase.R"))
# Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.
data <- rhc_data()
# Evaluate the fitted EIF at all eight observed combinations using the initial Q and g, producing eight values. Q is not updated here.
initial_d <- eif_values(data$initial_fit, data$cells)
# mapply reads a and x together from each group-table row. clever_h returns one weight per row, giving a vector of length four.
h <- mapply(function(a, x) clever_h(data$initial_fit, a, x), data$groups$a, data$groups$x)
# Join the group identifiers and H to display four rows. H may be positive or negative.
print(data.frame(data$groups[c("x", "a")], h = h))
# Multiply each observed combination’s empirical probability by its initial EIF value, then sum to obtain one empirical mean. No correction has yet been applied.
correction <- sum(data$cells$p * initial_d)
# Display the initial standardised risk difference and empirical EIF mean separately. A non-zero mean does not make individual residuals invalid.
cat("initial estimate =", data$initial_estimate, "; empirical EIF mean =", correction, "\n")
# Adjustment here uses sex alone; Chapter 8 constructs the AIPW correction and Q update.
```

<details><summary>Line-by-line explanation: input, output, dimensions and mathematical expressions</summary>

<div class="r-line-card" data-r-line="4">

#### R line 4

```r
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
```

<p>Input: constants or the current R environment. Output: <code>script_argument</code> · character vector · path or parameter string.</p>

commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.

</div>

<div class="r-line-card" data-r-line="6">

#### R line 6

```r
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
```

<p>Input: constants or the current R environment. Output: <code>source_files</code> · character vector · path or parameter string.</p>

sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.

</div>

<div class="r-line-card" data-r-line="8">

#### R line 8

```r
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch04.R")
```

<p>Input: <code>source_files</code> · character vector · path or parameter string; <code>script_argument</code> · character vector · path or parameter string. Output: <code>script_path</code> · character vector · path or parameter string.</p>

if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.

</div>

<div class="r-line-card" data-r-line="10">

#### R line 10

```r
stopifnot(length(script_path) == 1L, !is.na(script_path))
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.

</div>

<div class="r-line-card" data-r-line="12">

#### R line 12

```r
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: <code>project_root</code> · character vector · path or parameter string.</p>

dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.

</div>

<div class="r-line-card" data-r-line="14">

#### R line 14

```r
source(file.path(project_root, "shared", "R", "RHCCase.R"))
```

<p>Input: <code>project_root</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.

</div>

<div class="r-line-card" data-r-line="16">

#### R line 16

```r
data <- rhc_data()
```

<p>Input: constants or the current R environment. Output: <code>data</code> · list · Named container; each member has its own dimensions.</p>

Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.

</div>

<div class="r-line-card" data-r-line="18">

#### R line 18

```r
initial_d <- eif_values(data$initial_fit, data$cells)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>initial_d</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Evaluate the fitted EIF at all eight observed combinations using the initial Q and g, producing eight values. Q is not updated here.

$$
D_j=H_j(y_j-Q_j)+\tau(x_j)-\psi
$$


</div>

<div class="r-line-card" data-r-line="20">

#### R line 20

```r
h <- mapply(function(a, x) clever_h(data$initial_fit, a, x), data$groups$a, data$groups$x)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>h</code> · numeric vector · Length 4; aligned according to (x, a) group.</p>

mapply reads a and x together from each group-table row. clever_h returns one weight per row, giving a vector of length four.

$$
H(a,x)=a/g(x)-(1-a)/(1-g(x))
$$


</div>

<div class="r-line-card" data-r-line="22">

#### R line 22

```r
print(data.frame(data$groups[c("x", "a")], h = h))
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>h</code> · numeric vector · Length 4; aligned according to (x, a) group. Output: checks, displayed output or control flow.</p>

Join the group identifiers and H to display four rows. H may be positive or negative.

</div>

<div class="r-line-card" data-r-line="24">

#### R line 24

```r
correction <- sum(data$cells$p * initial_d)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>initial_d</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>correction</code> · numeric scalar (double) · Length 1.</p>

Multiply each observed combination’s empirical probability by its initial EIF value, then sum to obtain one empirical mean. No correction has yet been applied.

$$
P_n\widehat D=\sum_jp_j\widehat D_j
$$


</div>

<div class="r-line-card" data-r-line="26">

#### R line 26

```r
cat("initial estimate =", data$initial_estimate, "; empirical EIF mean =", correction, "\n")
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>correction</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Display the initial standardised risk difference and empirical EIF mean separately. A non-zero mean does not make individual residuals invalid.

</div>

</details>

</div>

## Next: verify correct derivatives in every admissible direction

We have explained the origins of the residual, inverse probability weight, signs and centring term, checking them numerically with real grouped counts. The next chapter pairs the candidate function with every admissible direction to complete gradient verification.


---

# Chapter 5: Why must we check every admissible direction?

Chapter 4 constructed $D$ using point replacement. Agreement on one path is insufficient: another function might give the same derivative on that path but fail along others. This chapter first covers every independent direction in the RHC joint frequency table, then explains what pathwise differentiability requires in general smooth models.

## 1. How many independent directions are available for eight probabilities?

Eight probabilities sum to 1, so admissible probability changes $h$ satisfy $\sum_{j=1}^8h_j=0$, leaving seven degrees of freedom. Every observation combination has positive probability in the original distribution, allowing the linear path $p_j(t)=p_j+th_j$; for sufficiently small $t$, all probabilities stay positive. The score is $S_j=h_j/p_j$, satisfying $\mathbb E_PS=0$.

Construct one direction for each of the first seven combinations: set its entry to 1, the eighth entry to −1, and all other entries to zero. This gives seven independent $h$ vectors. Every zero-sum $h$ is a linear combination of them. This adds a crucial step beyond checking one point direction: **it covers every admissible direction in the finite model.**

## 2. From paths to directional derivatives

A regular submodel $P_t$ passes through $P$ at $t=0$ and has a square-integrable score $S=\partial_t\log p_t|_0$. Write its directional derivative as:

$$
\dot\Psi_P(S)=\left.\frac d{dt}\Psi(P_t)\right|_0.
$$

> **Notation recap:** $P$ is the reference distribution; $S$ describes the path’s relative direction at its starting point. The quantity $\dot\Psi$ returns one scalar rate of change in that direction.

If, for every admissible $S$,

$$
\dot\Psi_P(S)=\mathbb E_P[D_PS],
$$

then $D$ is a gradient representer. The right-hand side is a probability-weighted inner product: the gradient stays fixed, the direction varies, and each pairing gives a different output. This resembles taking the dot product of a gradient with a direction for a familiar multivariable function, except that the coordinates here are probabilities or functions and the weights come from $P$.

## 3. Decompose a score into contributions from the three mechanisms

Taking logs turns the joint density’s product into a sum. Decompose $S$ as:

$$
\begin{aligned}
S_X&=\mathbb E[S\mid X],\\
S_A&=\mathbb E[S\mid A,X]-\mathbb E[S\mid X],\\
S_Y&=S-\mathbb E[S\mid A,X],\\
S&=S_X+S_A+S_Y.
\end{aligned}
$$

> **Notation recap:** A conditional expectation first averages within the specified group. Subtraction ensures $\mathbb E[S_A\mid X]=0$ and $\mathbb E[S_Y\mid A,X]=0$. The three terms therefore describe population, treatment and conditional outcome changes respectively, and are pairwise orthogonal under the $P$ inner product.

The conditional probability derivative satisfies $p_Y'=p_YS_Y$, so term-by-term differentiation gives $\dot Q=\sum_y y p_Y S_Y=\mathbb E[Y S_Y\mid a,x]$. Since $Q$ is fixed within the group and $\mathbb E[S_Y\mid a,x]=0$, subtracting $Q$ leaves this unchanged. Thus the conditional mean’s derivative along the $Y$ mechanism is $\dot Q(a,x)=\mathbb E[(Y-Q(a,x))S_Y\mid a,x]$. The outer population derivative is $\mathbb E[(\tau-\psi)S_X]$.

The target derivative is therefore the average over the common baseline population of the two arms’ conditional derivatives, plus the outer term. Multiplying the conditional expectation given $(a,x)$ by $p_X(x)\pi_a(x)$ and dividing by $\pi_a(x)$ restores the expression:

$$
\dot\Psi_P(S)=\mathbb E_P[H_g(Y-Q)S_Y]+\mathbb E_P[(\tau-\psi)S_X].
$$

> **Notation recap:** $H$ remains Chapter 4’s signed inverse treatment probability. There is no $S_A$ term because changing only $g$, while holding $Q$ and $p_X$ fixed, does not change the target.

## 4. Why does the full score give the same derivative?

Let $D_Y=H_g(Y-Q)$ and $D_X=\tau-\psi$. Given $A,X$, $D_Y$ has mean zero, so its pairings with $S_X,S_A$, which depend only on $A,X$, vanish. The term $D_X$ depends only on $X$, while $S_A,S_Y$ have conditional mean zero given $X$, so these cross-pairings also vanish. Expanding all six terms leaves only the previous section’s two terms:

$$
\mathbb E[(D_X+D_Y)(S_X+S_A+S_Y)]
=\mathbb E[D_XS_X]+\mathbb E[D_YS_Y]=\dot\Psi_P(S).
$$

> **Notation recap:** Conditional mean zero removes the cross inner products, leaving only the pairings of $D_X$ with $S_X$ and $D_Y$ with $S_Y$. This verifies that Chapter 4’s $D$ gives the correct derivative for every admissible $S$.

## Can we add a mean-zero function and still obtain correct derivatives?

The derivative along a pure treatment path $S_A$ should be zero. What happens in this direction if we arbitrarily add the mean-zero function $U=A-g(X)$ to $D$?

<div class="lab" data-lab="5"></div>
<div class="case-workbench" data-case-study="5"></div>

<details><summary>Answer and explanation</summary>

The original $D$ has a zero inner product with a treatment direction; the new function $D+bU$ pairs with $U$ to give $b\mathbb E U^2$. With non-zero $b$ and treatment probabilities strictly between 0 and 1, this is generally non-zero, so it cannot represent the correct derivative in a model allowing treatment changes. Mean zero is insufficient. Chapter 6 explains why a different model with fixed $g$ permits this extra component.

</details>

## Compare finite differences and gradient pairings in code

<div class="lesson-language-control"><label>Code language in this chapter <select data-lesson-language><option value="julia">Julia operations</option><option value="r">R operations</option></select></label><span>Use the same RHC data and the same mathematical objectives</span></div>

<div class="lesson-code-panel" data-lesson-panel="julia">


The webpage checks all eight point replacements; the script lists one point direction, seven independent contrast directions and directions that change individual mechanisms or several mechanisms together. On the same empirical table, it first recalculates the standardised difference from new joint probabilities, then takes a central finite difference. This and the $D$ pairing are independent calculations. The algebra covers all seven dimensions of the finite model; numerical checks test the implementation and compare finite-difference error at different step sizes.

The input is [the same original RHC CSV](../shared/data/public/rhc/source.csv). For reading and fitting definitions, see [RHCCase.jl](../shared/julia/RHCCase.jl). The complete [chapter script](julia/rhc_lessons/ch05.jl) is editable; all scripts can be run from other working directories, as long as the relative positions of en/ and shared/ are retained.

```sh
julia --startup-file=no --history-file=no en/julia/rhc_lessons/ch05.jl
```

```julia
# Public RHC cohort: X=sex, A=RHC receipt, Y=30-day death; all quantities are computed from original records.
# See BEGIN_CH05 in EIFCase.jl for numerical checks of general directions.
# Compute the gradient at the full empirical distribution; do not treat misspecified initial Q as the true model.
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
using .RHCCase
using Printf

data = rhc_data()
model = data.empirical_model
score = [(c.x-model.px[2]) + (c.a-model.g[c.x+1]) +
         (c.y-model.q[c.a+1,c.x+1]) for c in data.cells]
check = direction_check(data, score)
@printf("Mixed direction: difference quotient=%.9f, Gradient pairing=%.9f\n", check.finite_difference, check.pairing)
outcome_score = [c.y-model.q[c.a+1,c.x+1] for c in data.cells]
outcome_check = direction_check(data, outcome_score)
@printf("Outcome direction: difference quotient=%.9f, Gradient pairing=%.9f\n", outcome_check.finite_difference, outcome_check.pairing)
point = point_direction(data, 4) # The same (0,1,1) point direction as Chapter 2.
point_check = direction_check(data, point.score)
@printf("Same point direction: mean derivative=%.6f, Sex-standardised risk-difference derivative pairing=%.6f\n", point.derivative, point_check.pairing)

p = [c.p for c in data.cells]
for j in 1:7
    h = zeros(8)
    h[j] = 1.0
    h[8] = -1.0
    contrast = direction_check(data, h ./ p)
    @printf("Contrast direction %d: direct finite difference=%.9f, Gradient pairing=%.9f\n", j, contrast.finite_difference, contrast.pairing)
end
println("Finite-direction checks help find errors; the conclusion for all directions still relies on the proof in the text.")
```


<details><summary>View the source for this chapter’s mathematical functions</summary>

RHCCase reuses the following functions with counts from the source data. They contain no built-in values for the unknown population target.

```julia
function model_from_probabilities(cells, p)
    checked_probabilities(p); length(cells) == length(p) || throw(ArgumentError("Length mismatch"))
    px = [sum(p[i] for (i,c) in enumerate(cells) if c.x == x) for x in 0:1]
    mass = [sum(p[i] for (i,c) in enumerate(cells) if c.a == a && c.x == x) for a in 0:1, x in 0:1]
    q = [sum(p[i] * c.y for (i,c) in enumerate(cells) if c.a == a && c.x == x) /
         mass[a+1,x+1] for a in 0:1, x in 0:1]
    (; px, g=[mass[2,x+1] / px[x+1] for x in 0:1], q)
end

# Chapter 5: check the ATE gradient at the saturated empirical distribution; the initial fitted Q defines a different estimating function.
function direction_check(data, score; step::Real=1e-6)
    isfinite(step) && step > 0 || throw(ArgumentError("Positive finite difference step required"))
    p = [c.p for c in data.cells]
    plus = model_from_probabilities(data.cells, linear_path(p, score, step))
    minus = model_from_probabilities(data.cells, linear_path(p, score, -step))
    finite_difference = (ate(plus) - ate(minus)) / (2step)
    pairing = weighted_inner(p, eif_values(data.empirical_model, data.cells), score)
    (; finite_difference, pairing, difference=finite_difference - pairing)
end
```

</details>

<details><summary>Why do general models also require continuity of the directional derivative?</summary>

In a general $L^2$ model, the directional derivative must also be a continuous linear functional of the score. If $D$ is square-integrable, Cauchy–Schwarz gives $|\mathbb E DS|\le\sqrt{\mathbb E D^2}\sqrt{\mathbb E S^2}$, showing that small directions cannot produce unboundedly amplified pairings. A point mass in a continuous distribution need not be an admissible direction. The definition checks regular paths within the model, rather than arbitrary “distributional changes”. Some targets have no such finite-variance $D$ in a given model and therefore lack the pathwise differentiability used in this course.

</details>



</div>

<div class="lesson-code-panel" data-lesson-panel="r" hidden>

The input is [the same original public RHC CSV](../shared/data/public/rhc/source.csv). Shared definitions: [RHCCase.R](../shared/R/RHCCase.R) and [EIFCase.R](../shared/R/EIFCase.R). [Complete R script](R/ch05.R) uses base R and runs from any working directory if you retain the project directory structure.

```sh
Rscript --vanilla en/R/ch05.R
```

```r
# Chapter 5: checks in multiple directions
# Input: frozen original public RHC CSV; output: chapter objects and terminal checks; base R only.
# commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
# sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
# if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch05.R")
# length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.
stopifnot(length(script_path) == 1L, !is.na(script_path))
# dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
# source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.
source(file.path(project_root, "shared", "R", "RHCCase.R"))
# Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.
data <- rhc_data()
# Use a list representing the saturated model of the full empirical distribution as the mathematical reference point for checking gradients.
model <- data$empirical_model
# Extract the eight-cell table in x, a, y order, keeping subsequent vectors aligned.
cells <- data$cells
# A two-column index matrix selects individual entries from the R matrix. The first cbind column gives row a+1 and the second gives column x+1, producing eight values.
q_at_cell <- model$q[cbind(cells$a + 1L, cells$x + 1L)]
# Add the three centred mechanism directions element by element to obtain eight mixed-score values. Index g using x+1.
score <- (cells$x - model$px[2]) + (cells$a - model$g[cells$x + 1L]) + (cells$y - q_at_cell)
# Calculate a central difference quotient and gradient pairing along a valid probability path; return a list.
check <- direction_check(data, score)
# Keep only the centred residual of the outcome mechanism, producing eight values; hold the other two mechanisms fixed.
outcome_score <- cells$y - q_at_cell
# Check the pure outcome direction at the same empirical reference point.
outcome_check <- direction_check(data, outcome_score)
# Reuse Chapter 2’s point direction at (0,1,1), returning a list with the score and mean derivative.
point <- point_direction(data, 4L)
# Pair the same score with the derivative of the sex-standardised risk difference. This target differs from the mean.
point_check <- direction_check(data, point$score)
# Store the three checks by name in a list and print them separately, keeping derivatives of different targets distinct.
print(list(mixed = check, outcome = outcome_check, point = point_check))
# Read the eight empirical cell probabilities as elementwise divisors when converting h to a score.
p <- cells$p
# The for loop assigns j the values 1 to 7. The opening brace begins the loop body, constructing seven independent probability-mass transfer directions.
for (j in 1:7) {
# numeric(8) creates a numeric vector of eight zeros, newly created on each iteration.
  h <- numeric(8) 
# Using one-based [] indexing, set the mass increment in cell j to 1.
  h[j] <- 1
# Set the eighth increment to −1 to offset +1 in cell j, ensuring that h sums to 0.
  h[8] <- -1
# Elementwise / converts the mass direction h into a score. The shared function uses a fixed central-difference step on paths with strictly positive probabilities.
  contrast_check <- direction_check(data, h / p)
# Display the direct difference quotient and gradient pairing for this independent direction.
  cat("direction", j, "; finite difference =", contrast_check$finite_difference, "; pairing =", contrast_check$pairing, "\n")
# End the loop. These seven numerical checks help detect errors; the result for all directions still relies on the proof in the text.
}
```

<details><summary>Line-by-line explanation: input, output, dimensions and mathematical expressions</summary>

<div class="r-line-card" data-r-line="4">

#### R line 4

```r
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
```

<p>Input: constants or the current R environment. Output: <code>script_argument</code> · character vector · path or parameter string.</p>

commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.

</div>

<div class="r-line-card" data-r-line="6">

#### R line 6

```r
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
```

<p>Input: constants or the current R environment. Output: <code>source_files</code> · character vector · path or parameter string.</p>

sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.

</div>

<div class="r-line-card" data-r-line="8">

#### R line 8

```r
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch05.R")
```

<p>Input: <code>source_files</code> · character vector · path or parameter string; <code>script_argument</code> · character vector · path or parameter string. Output: <code>script_path</code> · character vector · path or parameter string.</p>

if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.

</div>

<div class="r-line-card" data-r-line="10">

#### R line 10

```r
stopifnot(length(script_path) == 1L, !is.na(script_path))
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.

</div>

<div class="r-line-card" data-r-line="12">

#### R line 12

```r
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: <code>project_root</code> · character vector · path or parameter string.</p>

dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.

</div>

<div class="r-line-card" data-r-line="14">

#### R line 14

```r
source(file.path(project_root, "shared", "R", "RHCCase.R"))
```

<p>Input: <code>project_root</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.

</div>

<div class="r-line-card" data-r-line="16">

#### R line 16

```r
data <- rhc_data()
```

<p>Input: constants or the current R environment. Output: <code>data</code> · list · Named container; each member has its own dimensions.</p>

Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.

</div>

<div class="r-line-card" data-r-line="18">

#### R line 18

```r
model <- data$empirical_model
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>model</code> · list · Named container; each member has its own dimensions.</p>

Use a list representing the saturated model of the full empirical distribution as the mathematical reference point for checking gradients.

</div>

<div class="r-line-card" data-r-line="20">

#### R line 20

```r
cells <- data$cells
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>cells</code> · data.frame · 8 rows × 5 columns (x, a, y, n, p).</p>

Extract the eight-cell table in x, a, y order, keeping subsequent vectors aligned.

</div>

<div class="r-line-card" data-r-line="22">

#### R line 22

```r
q_at_cell <- model$q[cbind(cells$a + 1L, cells$x + 1L)]
```

<p>Input: <code>model</code> · list · Named container; each member has its own dimensions; <code>cells</code> · data.frame · 8 rows × 5 columns (x, a, y, n, p). Output: <code>q_at_cell</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

A two-column index matrix selects individual entries from the R matrix. The first cbind column gives row a+1 and the second gives column x+1, producing eight values.

$$
q_j=Q(a_j,x_j)
$$


</div>

<div class="r-line-card" data-r-line="24">

#### R line 24

```r
score <- (cells$x - model$px[2]) + (cells$a - model$g[cells$x + 1L]) + (cells$y - q_at_cell)
```

<p>Input: <code>cells</code> · data.frame · 8 rows × 5 columns (x, a, y, n, p); <code>model</code> · list · Named container; each member has its own dimensions; <code>q_at_cell</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>score</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Add the three centred mechanism directions element by element to obtain eight mixed-score values. Index g using x+1.

$$
S=(X-p_1)+(A-g(X))+(Y-Q(A,X))
$$


</div>

<div class="r-line-card" data-r-line="26">

#### R line 26

```r
check <- direction_check(data, score)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>score</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>check</code> · list · Named container; each member has its own dimensions.</p>

Calculate a central difference quotient and gradient pairing along a valid probability path; return a list.

$$
\dot\Psi\approx[\Psi(P_h)-\Psi(P_{-h})]/(2h)
$$


</div>

<div class="r-line-card" data-r-line="28">

#### R line 28

```r
outcome_score <- cells$y - q_at_cell
```

<p>Input: <code>cells</code> · data.frame · 8 rows × 5 columns (x, a, y, n, p); <code>q_at_cell</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>outcome_score</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Keep only the centred residual of the outcome mechanism, producing eight values; hold the other two mechanisms fixed.

$$
S_Y=Y-Q(A,X)
$$


</div>

<div class="r-line-card" data-r-line="30">

#### R line 30

```r
outcome_check <- direction_check(data, outcome_score)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>outcome_score</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>outcome_check</code> · list · Named container; each member has its own dimensions.</p>

Check the pure outcome direction at the same empirical reference point.

</div>

<div class="r-line-card" data-r-line="32">

#### R line 32

```r
point <- point_direction(data, 4L)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>point</code> · list · Named container describing a point direction.</p>

Reuse Chapter 2’s point direction at (0,1,1), returning a list with the score and mean derivative.

</div>

<div class="r-line-card" data-r-line="34">

#### R line 34

```r
point_check <- direction_check(data, point$score)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>point</code> · list · Named container describing a point direction. Output: <code>point_check</code> · list · Named container; each member has its own dimensions.</p>

Pair the same score with the derivative of the sex-standardised risk difference. This target differs from the mean.

</div>

<div class="r-line-card" data-r-line="36">

#### R line 36

```r
print(list(mixed = check, outcome = outcome_check, point = point_check))
```

<p>Input: <code>check</code> · list · Named container; each member has its own dimensions; <code>outcome_check</code> · list · Named container; each member has its own dimensions; <code>point_check</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Store the three checks by name in a list and print them separately, keeping derivatives of different targets distinct.

</div>

<div class="r-line-card" data-r-line="38">

#### R line 38

```r
p <- cells$p
```

<p>Input: <code>cells</code> · data.frame · 8 rows × 5 columns (x, a, y, n, p). Output: <code>p</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Read the eight empirical cell probabilities as elementwise divisors when converting h to a score.

</div>

<div class="r-line-card" data-r-line="40">

#### R line 40

```r
for (j in 1:7) {
```

<p>Input: constants or the current R environment. Output: <code>j</code> · integer scalar · Length 1.</p>

The for loop assigns j the values 1 to 7. The opening brace begins the loop body, constructing seven independent probability-mass transfer directions.

</div>

<div class="r-line-card" data-r-line="42">

#### R line 42

```r
  h <- numeric(8) 
```

<p>Input: constants or the current R environment. Output: <code>h</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

numeric(8) creates a numeric vector of eight zeros, newly created on each iteration.

</div>

<div class="r-line-card" data-r-line="44">

#### R line 44

```r
  h[j] <- 1
```

<p>Input: <code>h</code> · numeric vector · Length 8; aligned according to (x, a, y) grid; <code>j</code> · integer scalar · Length 1. Output: <code>h</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Using one-based [] indexing, set the mass increment in cell j to 1.

</div>

<div class="r-line-card" data-r-line="46">

#### R line 46

```r
  h[8] <- -1
```

<p>Input: <code>h</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>h</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Set the eighth increment to −1 to offset +1 in cell j, ensuring that h sums to 0.

$$
\sum_jh_j=0
$$


</div>

<div class="r-line-card" data-r-line="48">

#### R line 48

```r
  contrast_check <- direction_check(data, h / p)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>h</code> · numeric vector · Length 8; aligned according to (x, a, y) grid; <code>p</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>contrast_check</code> · list · Named container; each member has its own dimensions.</p>

Elementwise / converts the mass direction h into a score. The shared function uses a fixed central-difference step on paths with strictly positive probabilities.

$$
S_j=h_j/p_j
$$


</div>

<div class="r-line-card" data-r-line="50">

#### R line 50

```r
  cat("direction", j, "; finite difference =", contrast_check$finite_difference, "; pairing =", contrast_check$pairing, "\n")
```

<p>Input: <code>j</code> · integer scalar · Length 1; <code>contrast_check</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the direct difference quotient and gradient pairing for this independent direction.

</div>

<div class="r-line-card" data-r-line="52">

#### R line 52

```r
}
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

End the loop. These seven numerical checks help detect errors; the result for all directions still relies on the proof in the text.

</div>

</details>

</div>

## Next: use model constraints and projection to identify the EIF

We have verified the gradient identity for every direction in the finite nonparametric model and explained the general definition. The next chapter incorporates model constraints into spaces and projection calculations to select the efficient representer; Chapter 7 then develops the estimation-error expansion.


---

# Chapter 6: Identify the efficient influence function using tangent spaces and projection

Chapter 5 required the inner product of $D$ with each admissible score to give the correct derivative. Now ask: what determines admissible directions? If a model restricts one mechanism and several functions pass the pairing checks, which gives the efficiency bound? We retain the same RHC joint frequency table and calculate projections using function values and probability weights at each observation combination.

## 1. Calculate inner products and lengths using probability weights

A function on these eight observation combinations can be written $f=(f_1,\ldots,f_8)$, where entry $j$ is its output at combination $j$. Using empirical reference probabilities $p_j=n_j/5735$ from the real cohort, define:

$$
\langle f,h\rangle_P=\sum_{j=1}^8p_jf_jh_j,
\qquad \|f\|_P^2=\sum_jp_jf_j^2.
$$

> **Notation recap:** An inner product multiplies row by row, then averages under $P$; the squared norm is the inner product with itself. Each observation combination has weight $p_j=n_j/5735$. Positive probabilities ensure a non-zero vector has positive squared length.

The space of mean-zero square-integrable functions is denoted $L^2_0(P)$. Here each function has eight values, forming an eight-dimensional vector; mean zero imposes one linear constraint, so this space has dimension seven. General function spaces retain the same weighted-average and length ideas while allowing infinitely many values.

## 2. How do model constraints determine the tangent space?

The tangent space is the closure of admissible scores and their linear combinations. Taking the closure includes limits of sequences of these directions. In finite dimensions, no additional limits are needed for the calculation.

- **The full observational model** allows $p_X,g,Q$ to vary, giving $T=L^2_0(P)$, of dimension seven.
- **This chapter’s hypothetical exercise** fixes $g$ at its current empirical value and allows only $p_X,Q$ to vary, giving a five-dimensional $T$. This exercise lets us compare the effects of model constraints; the course’s observational RHC analysis and Chapter 8 use a model with unknown $g$.

Five basis directions are:

$$
B_X=X-p_X(1),\qquad
B_{xa}=\mathbf1\{X=x,A=a\}[Y-Q(a,x)]\quad(x,a\in\{0,1\}).
$$

> **Notation recap:** The first basis function changes population composition; each of the next four changes one group’s conditional outcome distribution. All five basis functions have mean zero and are pairwise orthogonal: different $(a,x)$ columns cannot be non-zero together, and conditional mean-zero residuals make each orthogonal to $B_X$.

## 3. Why can there be several gradients?

Let $U=A-g(X)$. It has mean zero given $X$ and is orthogonal to every $B$: for $B_X$, average $U$ given $X$ first; for $B_{xa}$, average the residual given $A,X$ first. Both give zero. Thus $D_b=D^*+bU$ pairs with every admissible $S$ exactly as $D^*$ does. Here $b$ is any real number, and $U$ is not an admissible treatment direction in this fixed-$g$ model.

Which components may be added depends on the model. If $g$ is unknown, $U$ is admissible too, and the target’s pure-treatment derivative is zero. The new candidate instead gives $b\mathbb E U^2$, failing for non-zero $b$.

## 4. How does projection select the gradient of smallest norm?

Orthogonal projection finds the function in $T$ closest to candidate $F$. Start with one column $B$:

$$
\begin{aligned}
\|F-cB\|^2&=\|F\|^2-2c\langle F,B\rangle+c^2\|B\|^2\\
&=\|F\|^2-\frac{\langle F,B\rangle^2}{\|B\|^2}
+\|B\|^2\left(c-\frac{\langle F,B\rangle}{\|B\|^2}\right)^2.
\end{aligned}
$$

> **Notation recap:** Here $c$ is the projection coefficient to choose, and $B$ a non-zero basis column. The final square is non-negative; setting its bracket to zero minimises the expression.

First establish $D^*\in T$: for binary $X$, $D_X=[\tau(1)-\tau(0)]B_X$, and $D_Y=\sum_{x,a}H_g(a,x)B_{xa}$. These are the previous section’s five columns, with a coefficient fixed within each group, so $D^*$ is indeed their linear combination. Together with $U$’s orthogonality to every column, this justifies saying that projection retains $D^*$ and removes $bU$.

When the five columns are orthogonal, calculate $\langle F,B_k\rangle/\|B_k\|^2$ for each, multiply by its $B_k$, and add. A programme can also subtract existing projections column by column to handle a general independent basis. For $F=D^*+bU$, the projection is exactly $D^*$ and residual $bU$ is orthogonal to $T$, hence:

$$
\|D^*+bU\|^2=\|D^*\|^2+2b\langle D^*,U\rangle+b^2\|U\|^2
=\|D^*\|^2+b^2\|U\|^2.
$$

> **Notation recap:** Orthogonality makes the cross-term zero; the remaining term is non-negative. The unique function in $T$ representing the target derivative is the canonical gradient, also called the efficient influence function (EIF). Its “shortest” length uses $P$ weights and connects to variance in Chapter 7.

## How do projection and norm change when the orthogonal component increases?

Change $b$ from 1 to 2: does the projection change? What happens to the removed component’s squared length?

<div class="lab" data-lab="6"></div>
<div class="case-workbench" data-case-study="6"></div>

<details><summary>Answer and explanation</summary>

The projection remains $D^*$ because the added component is still orthogonal. The residual changes from $U$ to $2U$, quadrupling its squared length. The table calculates the projection onto the five basis directions, orthogonal inner products and norm decomposition. Any two-axis picture is schematic; this experiment’s numerical objects are always eight-row column vectors.

</details>

## Code workbook: input, operation and output

<div class="lesson-language-control"><label>Code language in this chapter <select data-lesson-language><option value="julia">Julia operations</option><option value="r">R operations</option></select></label><span>Use the same RHC data and the same mathematical objectives</span></div>

<div class="lesson-code-panel" data-lesson-panel="julia">

The input is [the same original RHC CSV](../shared/data/public/rhc/source.csv). For reading and fitting definitions, see [RHCCase.jl](../shared/julia/RHCCase.jl). The complete [chapter script](julia/rhc_lessons/ch06.jl) is editable; all scripts can be run from other working directories, as long as the relative positions of en/ and shared/ are retained.

```sh
julia --startup-file=no --history-file=no en/julia/rhc_lessons/ch06.jl
```

```julia
# Public RHC cohort: X=sex, A=RHC receipt, Y=30-day death; all quantities are computed from original records.
# See BEGIN_CH06 in EIFCase.jl for the full projection implementation; no projection package is used.
# Only this exercise assumes estimated g is known and fixed; the final TMLE still learns g from supplied counts.
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
using .RHCCase
using Printf

data = rhc_data()
demo = projection_demo(data; b=1.0)
@printf("Candidate squared norm=%.6f, Projection squared norm=%.6f, Residual squared norm=%.6f\n", demo.candidate_second_moment, demo.projection_second_moment, demo.residual_second_moment)
@printf("Maximum difference between projection and canonical gradient=%.3g\n", maximum(abs.(demo.projection-demo.canonical)))
```


<details><summary>View the source for this chapter’s mathematical functions</summary>

RHCCase reuses the following functions with counts from the source data. They contain no built-in values for the unknown population target.

```julia
function weighted_projection(p, f, basis)
    checked_probabilities(p); checked_vector(f, length(p))
    orthonormal = Vector{Float64}[]
    for b in basis
        checked_vector(b, length(p))
        original_norm = sqrt(weighted_inner(p, b, b))
        original_norm > 0 || throw(ArgumentError("Basis must have full column rank"))
        v = Float64.(b) / original_norm
        for pass in 1:2, u in orthonormal
            v -= weighted_inner(p, v, u) * u
        end
        vnorm = sqrt(weighted_inner(p, v, v))
        vnorm > 1e-12 || throw(ArgumentError("Basis is rank deficient at numerical precision"))
        push!(orthonormal, v / vnorm)
    end
    result = zeros(length(p))
    for u in orthonormal
        result += weighted_inner(p, f, u) * u
    end
    result
end

# Chapter 6: only this exercise treats the count-based g as known and fixed; the final TMLE still learns g from the counts.
function projection_demo(data; b::Real=1.0)
    isfinite(b) || throw(ArgumentError("Finite candidate coefficient required"))
    model = data.empirical_model
    p = [c.p for c in data.cells]
    canonical = eif_values(model, data.cells)
    u = [c.a - model.g[c.x+1] for c in data.cells]
    basis = [[c.x - model.px[2] for c in data.cells]]
    append!(basis, [[c.x == x && c.a == a ? c.y - model.q[a+1,x+1] : 0.0
                     for c in data.cells] for x in 0:1 for a in 0:1])
    candidate = canonical + b * u
    projection = weighted_projection(p, candidate, basis)
    residual = candidate - projection
    (; basis, canonical, u, candidate, projection, residual,
       candidate_second_moment=weighted_inner(p, candidate, candidate),
       projection_second_moment=weighted_inner(p, projection, projection),
       residual_second_moment=weighted_inner(p, residual, residual))
end
```

</details>

</div>

<div class="lesson-code-panel" data-lesson-panel="r" hidden>

The input is [the same original public RHC CSV](../shared/data/public/rhc/source.csv). Shared definitions: [RHCCase.R](../shared/R/RHCCase.R) and [EIFCase.R](../shared/R/EIFCase.R). [Complete R script](R/ch06.R) uses base R and runs from any working directory if you retain the project directory structure.

```sh
Rscript --vanilla en/R/ch06.R
```

```r
# Chapter 6: projection in the known-g exercise
# Input: frozen original public RHC CSV; output: chapter objects and terminal checks; base R only.
# commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
# sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
# if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch06.R")
# length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.
stopifnot(length(script_path) == 1L, !is.na(script_path))
# dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
# source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.
source(file.path(project_root, "shared", "R", "RHCCase.R"))
# Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.
data <- rhc_data()
# In this exercise, keep empirical g fixed and let b specify coefficients of candidate additional components. Perform the weighted linear-algebra projection manually and return a list.
demo <- projection_demo(data, b = 1)
# Display three weighted squared norms to check the Pythagorean decomposition. Each scalar has units of squared probability differences.
cat("candidate norm squared =", demo$candidate_second_moment, "; projection =", demo$projection_second_moment, "; residual =", demo$residual_second_moment, "\n")
# Subtract the two eight-element vectors, take absolute values with abs, then use max to reduce them to one maximum error.
projection_error <- max(abs(demo$projection - demo$canonical))
# Display the maximum numerical difference between the projection and the canonical gradient in this model with fixed g.
cat("maximum projection error =", projection_error, "\n")
# This exercise’s five-dimensional known-g model differs from the seven-dimensional observational model with unknown g; the final TMLE still learns g from data.
```

<details><summary>Line-by-line explanation: input, output, dimensions and mathematical expressions</summary>

<div class="r-line-card" data-r-line="4">

#### R line 4

```r
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
```

<p>Input: constants or the current R environment. Output: <code>script_argument</code> · character vector · path or parameter string.</p>

commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.

</div>

<div class="r-line-card" data-r-line="6">

#### R line 6

```r
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
```

<p>Input: constants or the current R environment. Output: <code>source_files</code> · character vector · path or parameter string.</p>

sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.

</div>

<div class="r-line-card" data-r-line="8">

#### R line 8

```r
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch06.R")
```

<p>Input: <code>source_files</code> · character vector · path or parameter string; <code>script_argument</code> · character vector · path or parameter string. Output: <code>script_path</code> · character vector · path or parameter string.</p>

if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.

</div>

<div class="r-line-card" data-r-line="10">

#### R line 10

```r
stopifnot(length(script_path) == 1L, !is.na(script_path))
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.

</div>

<div class="r-line-card" data-r-line="12">

#### R line 12

```r
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: <code>project_root</code> · character vector · path or parameter string.</p>

dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.

</div>

<div class="r-line-card" data-r-line="14">

#### R line 14

```r
source(file.path(project_root, "shared", "R", "RHCCase.R"))
```

<p>Input: <code>project_root</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.

</div>

<div class="r-line-card" data-r-line="16">

#### R line 16

```r
data <- rhc_data()
```

<p>Input: constants or the current R environment. Output: <code>data</code> · list · Named container; each member has its own dimensions.</p>

Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.

</div>

<div class="r-line-card" data-r-line="18">

#### R line 18

```r
demo <- projection_demo(data, b = 1)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>demo</code> · list · Named container; each member has its own dimensions.</p>

In this exercise, keep empirical g fixed and let b specify coefficients of candidate additional components. Perform the weighted linear-algebra projection manually and return a list.

$$
f=D+b(A-g(X))
$$


</div>

<div class="r-line-card" data-r-line="20">

#### R line 20

```r
cat("candidate norm squared =", demo$candidate_second_moment, "; projection =", demo$projection_second_moment, "; residual =", demo$residual_second_moment, "\n")
```

<p>Input: <code>demo</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display three weighted squared norms to check the Pythagorean decomposition. Each scalar has units of squared probability differences.

$$
\lVert f\rVert^2=\lVert\Pi f\rVert^2+\lVert f-\Pi f\rVert^2
$$


</div>

<div class="r-line-card" data-r-line="22">

#### R line 22

```r
projection_error <- max(abs(demo$projection - demo$canonical))
```

<p>Input: <code>demo</code> · list · Named container; each member has its own dimensions. Output: <code>projection_error</code> · numeric scalar (double) · Length 1.</p>

Subtract the two eight-element vectors, take absolute values with abs, then use max to reduce them to one maximum error.

</div>

<div class="r-line-card" data-r-line="24">

#### R line 24

```r
cat("maximum projection error =", projection_error, "\n")
```

<p>Input: <code>projection_error</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Display the maximum numerical difference between the projection and the canonical gradient in this model with fixed g.

</div>

</details>

</div>



<details><summary>Does allowing g to vary change the EIF?</summary>

The gradient representer is unique in the full model’s seven-dimensional mean-zero space. Fixing $g$ permits additions from the orthogonal complement of $T$, but the EIF remains the projection. For this ATE form, $D^*$ already has no treatment-score component, so its EIF and variance are identical in these two models. If $Q$ were fixed and only $p_X,g$ could vary, the canonical gradient would instead be just $\tau(X)-\psi$. Model constraints affect the canonical gradient through admissible directions.

</details>

## Next: connect the EIF to estimation error and variance

We have used the cohort’s empirical probability weights to work through model constraints, projection and the minimum-norm argument. Starting with the familiar sample mean, the next chapter connects gradients to estimation error and explains the required conditions.


---

# Chapter 7: How does the EIF describe estimation error and determine efficiency bounds?

The preceding chapters studied how a target changes when its distribution changes. Researchers use a sample to estimate a fixed true target. This chapter connects distributional gradients to estimation error and explains when EIF variance can be called an efficiency lower bound.

## 1. Express mean estimation error exactly as an IF average

Suppose $Y_1,\ldots,Y_n$ are independent and identically distributed (iid), with true risk $\mu_0=\mathbb E_0Y$. Subtract the same mean term by term:

$$
\begin{aligned}
\widehat\mu-\mu_0&=\frac1n\sum_iY_i-\frac1n\sum_i\mu_0\\
&=\frac1n\sum_i(Y_i-\mu_0).
\end{aligned}
$$

> **Notation recap:** Averaging $n$ identical copies of $\mu_0$ still gives $\mu_0$; the bracket contains the mean IF derived in Chapter 1. Centring at the unknown true mean makes the right side the difference between the estimate and the true mean. Centring instead at the sample mean gives the separate identity $n^{-1}\sum_i(Y_i-\widehat\mu)=0$.

This expansion is exact; general targets usually have an additional remainder. The 5735 RHC patients give current sample risk 1918/5735, while the true population risk remains unknown. We therefore cannot calculate the true estimation error from this sample alone.

## 2. When can an IF average approximate a general estimator’s error?

An estimator is asymptotically linear if it satisfies:

$$
\widehat\psi-\psi_0=\frac1n\sum_i\phi_0(O_i)+r_n,
\quad\mathbb E_0\phi_0=0,\quad
r_n=o_P(n^{-1/2}).
$$

> **Notation recap:** $\phi_0$ is the estimator’s influence function and $\psi_0$ the population target; $o_P(n^{-1/2})$ means convergence in probability to zero after multiplication by $\sqrt n$. Merely having $r_n\to0$ is insufficient: it must be smaller than the leading sampling-error scale $1/\sqrt n$.

For example, multiplying $1/n$ by $\sqrt n$ gives a sequence tending to zero, while multiplying $1/\sqrt n$ by $\sqrt n$ gives 1. Both sequences tend to zero, but only the first is negligible relative to the first-order sampling error. For a specific estimator, this remainder requirement must be proved. Chapter 8 shows how a second-order remainder requires the fitting errors of $Q,g$ to be jointly small enough.

## 3. Why divide variance by n?

When independent mean-zero contributions are added, cross-covariances vanish, so:

$$
\operatorname{Var}_0\left(\frac1n\sum_i\phi_0(O_i)\right)
=\frac1{n^2}\sum_i\operatorname{Var}_0(\phi_0)
=\frac{\operatorname{Var}_0(\phi_0)}n.
$$

> **Notation recap:** Variance scales by squared weights; $n$ terms share the same variance, giving $n/n^2$. The standard error (SE) is the square root of variance, so quadrupling sample size halves the first-order SE.

With finite second moments and suitable conditions, the central limit theorem (CLT) makes $\sqrt n(\widehat\psi-\psi_0)$ converge to a normal distribution with mean zero and variance $\mathbb E_0\phi_0^2$. Sampling dependence, such as clustering by hospital, requires the corresponding variance theory.

## 4. Which estimators does the efficiency lower bound apply to?

A regular estimator must have stable first-order behaviour under local smooth perturbations, excluding rules that perform exceptionally well only at an isolated distribution. For regular asymptotically linear estimators, influence function $\phi$ must represent all the target’s path derivatives. Chapter 6’s projection therefore gives:

$$
\phi=D^*+u,\quad u\perp T,\qquad
\mathbb E\phi^2=\mathbb E(D^*)^2+\mathbb E u^2\ge\mathbb E(D^*)^2.
$$

> **Notation recap:** $D^*$ is the canonical gradient for the specified statistical model; $u$ is an additional component orthogonal to every admissible direction. The norm decomposition becomes a variance decomposition because the functions are centred. The efficiency bound concerns the first-order variance of these regular estimators; finite-sample mean squared error also depends on bias and higher-order terms.

Thus $\operatorname{Var}_0(D^*)/n$ gives the leading variance of estimation error when $\phi$ equals $D^*$ and the remainder is small enough. In real data, $D^*$ is unknown and must be estimated using consistently fitted functions.

## 5. Decompose the variance of the risk difference EIF

In Chapter 4, $D^*=D_X+D_Y$; conditional mean-zero residuals remove the cross-term. For binary $Y$, the conditional variance is $Q(1-Q)$. First average over the two observed treatment arms; one treatment probability cancels from each squared weight:

$$
\begin{aligned}
\mathbb E[D_Y^2\mid X]&=g\frac{Q_1(1-Q_1)}{g^2}+(1-g)\frac{Q_0(1-Q_0)}{(1-g)^2}\\
&=\frac{Q_1(1-Q_1)}g+\frac{Q_0(1-Q_0)}{1-g}.
\end{aligned}
$$

> **Notation recap:** Here $Q_a=Q(a,X)$ and $g=g(X)$; leading factors $g$ and $1-g$ are the actual probabilities of the two arms, and the squared denominators come from $H^2$. Finally averaging over $X$ gives:

$$
V(P)=\operatorname{Var}_P\{Q(1,X)-Q(0,X)\}
+\mathbb E_P\left[\frac{Q(1,X)[1-Q(1,X)]}{g(X)}
+\frac{Q(0,X)[1-Q(0,X)]}{1-g(X)}\right].
$$

> **Notation recap:** $V$ is the variance of a single gradient contribution, in squared probability-difference units. Divide this variance by $n$ to obtain the estimator’s first-order variance. As $g$ approaches a boundary, less information is available under the rare treatment and the reciprocal term may increase; the target itself can remain unchanged.

<div class="lab" data-lab="7"></div>

Predict first: if only $n$ quadruples, what happens to $V$, the target and SE?

<details><summary>Answer and explanation</summary>

The reference distribution stays fixed, so $V$ and the target do not change; $V/n$ becomes one quarter as large, and SE halves. The experiment’s curves fix $p_X,Q$ from the real empirical table and hypothetically vary $g$ to compare theoretical information in the specified reference model.

</details>

## 6. Calculate the empirical variance of initial fitted influence-function values

Chapter 4’s initial $Q$ uses only $A$, so its fitted influence-function values $F$ need not have sample average zero. Use the empirical variance definition from the Chapter 0 TMLE overview: subtract the sample average, square, and average over $n=5735$ patients. The denominator is still $n$:

$$
P_n(F-P_nF)^2=P_nF^2-(P_nF)^2.
$$

> **Notation recap:** $P_nf$ averages $f$ values over the same 5735 patients, while $P_nF^2$ squares first and then averages; this differs from averaging first and then squaring. Switching to $D^*$ at the saturated empirical table also changes $Q$ and the target, producing another function. Here $g$ still uses empirical within-sex treatment proportions.

<div class="case-workbench" data-case-study="7"></div>
## Code workbook: input, operation and output

<div class="lesson-language-control"><label>Code language in this chapter <select data-lesson-language><option value="julia">Julia operations</option><option value="r">R operations</option></select></label><span>Use the same RHC data and the same mathematical objectives</span></div>

<div class="lesson-code-panel" data-lesson-panel="julia">

The input is [the same original RHC CSV](../shared/data/public/rhc/source.csv). For reading and fitting definitions, see [RHCCase.jl](../shared/julia/RHCCase.jl). The complete [chapter script](julia/rhc_lessons/ch07.jl) is editable; all scripts can be run from other working directories, as long as the relative positions of en/ and shared/ are retained.

```sh
julia --startup-file=no --history-file=no en/julia/rhc_lessons/ch07.jl
```

```julia
# Public RHC cohort: X=sex, A=RHC receipt, Y=30-day death; all quantities are computed from original records.
# See BEGIN_CH07 in EIFCase.jl for weighted variance and SE calculations.
# Both functions are averaged over the current frequency table; neither is an efficiency bound at a known true distribution.
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
using .RHCCase
using Printf

data = rhc_data()
initial_values = eif_values(data.initial_fit, data.cells)
initial_moments = if_variance(data, initial_values)
@printf("Initial fitted function: mean=%.8f, second_moment=%.8f, variance=%.8f, se=%.8f (%.4f percentage points)\n", initial_moments.mean, initial_moments.second_moment, initial_moments.variance, initial_moments.se, 100initial_moments.se)

saturated_values = eif_values(data.empirical_model, data.cells)
saturated_moments = if_variance(data, saturated_values)
@printf("Saturated empirical-distribution function: mean=%.8f, second_moment=%.8f, variance=%.8f, se=%.8f (%.4f percentage points)\n", saturated_moments.mean, saturated_moments.second_moment, saturated_moments.variance, saturated_moments.se, 100saturated_moments.se)
println("Both are empirical quantities for this table, rather than an efficiency bound at a known true distribution or the actual estimation error of one sample.")
println("Recompute after updating Q in Chapter 8; neither SE is the updated SE.")
```


<details><summary>View the source for this chapter’s mathematical functions</summary>

RHCCase reuses the following functions with counts from the source data. They contain no built-in values for the unknown population target.

```julia
function if_variance(data, values=eif_values(data.empirical_model, data.cells))
    checked_vector(values, length(data.cells))
    mean = sum(c.p * values[i] for (i,c) in enumerate(data.cells))
    second_moment = sum(c.p * values[i]^2 for (i,c) in enumerate(data.cells))
    variance = sum(c.p * (values[i] - mean)^2 for (i,c) in enumerate(data.cells))
    (; mean, second_moment, variance, se=sqrt(variance / data.n))
end
```

</details>

</div>

<div class="lesson-code-panel" data-lesson-panel="r" hidden>

The input is [the same original public RHC CSV](../shared/data/public/rhc/source.csv). Shared definitions: [RHCCase.R](../shared/R/RHCCase.R) and [EIFCase.R](../shared/R/EIFCase.R). [Complete R script](R/ch07.R) uses base R and runs from any working directory if you retain the project directory structure.

```sh
Rscript --vanilla en/R/ch07.R
```

```r
# Chapter 7: empirical IF variance and SE
# Input: frozen original public RHC CSV; output: chapter objects and terminal checks; base R only.
# commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
# sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
# if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch07.R")
# length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.
stopifnot(length(script_path) == 1L, !is.na(script_path))
# dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
# source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.
source(file.path(project_root, "shared", "R", "RHCCase.R"))
# Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.
data <- rhc_data()
# Evaluate the EIF under the initial fitted Q and g at eight observed combinations, returning an eight-element vector.
initial_values <- eif_values(data$initial_fit, data$cells)
# Calculate the mean, second moment, centred variance and SE using the current cell probabilities; return a list.
initial_moments <- if_variance(data, initial_values)
# Evaluate eight EIF values under the saturated model of the full empirical distribution, distinct from the initial Q fitted using A alone.
saturated_values <- eif_values(data$empirical_model, data$cells)
# Use the same probability weights to obtain a second list of empirical moments.
saturated_moments <- if_variance(data, saturated_values)
# Display both sets of empirical means, second moments, variances and SEs by name. These are not known efficiency bounds at P0.
print(list(initial = initial_moments, saturated = saturated_moments))
# Multiply each standard error by 100 to convert a probability difference to percentage points. A variance would instead be multiplied by 10,000 to express it in squared percentage points.
cat("initial SE in percentage points =", 100 * initial_moments$se, "; saturated =", 100 * saturated_moments$se, "\n")
# These quantities are not the actual estimation error in one sample; recompute SE after updating Q in Chapter 8.
```

<details><summary>Line-by-line explanation: input, output, dimensions and mathematical expressions</summary>

<div class="r-line-card" data-r-line="4">

#### R line 4

```r
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
```

<p>Input: constants or the current R environment. Output: <code>script_argument</code> · character vector · path or parameter string.</p>

commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.

</div>

<div class="r-line-card" data-r-line="6">

#### R line 6

```r
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
```

<p>Input: constants or the current R environment. Output: <code>source_files</code> · character vector · path or parameter string.</p>

sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.

</div>

<div class="r-line-card" data-r-line="8">

#### R line 8

```r
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch07.R")
```

<p>Input: <code>source_files</code> · character vector · path or parameter string; <code>script_argument</code> · character vector · path or parameter string. Output: <code>script_path</code> · character vector · path or parameter string.</p>

if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.

</div>

<div class="r-line-card" data-r-line="10">

#### R line 10

```r
stopifnot(length(script_path) == 1L, !is.na(script_path))
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.

</div>

<div class="r-line-card" data-r-line="12">

#### R line 12

```r
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: <code>project_root</code> · character vector · path or parameter string.</p>

dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.

</div>

<div class="r-line-card" data-r-line="14">

#### R line 14

```r
source(file.path(project_root, "shared", "R", "RHCCase.R"))
```

<p>Input: <code>project_root</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.

</div>

<div class="r-line-card" data-r-line="16">

#### R line 16

```r
data <- rhc_data()
```

<p>Input: constants or the current R environment. Output: <code>data</code> · list · Named container; each member has its own dimensions.</p>

Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.

</div>

<div class="r-line-card" data-r-line="18">

#### R line 18

```r
initial_values <- eif_values(data$initial_fit, data$cells)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>initial_values</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Evaluate the EIF under the initial fitted Q and g at eight observed combinations, returning an eight-element vector.

</div>

<div class="r-line-card" data-r-line="20">

#### R line 20

```r
initial_moments <- if_variance(data, initial_values)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>initial_values</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>initial_moments</code> · list · Named container; each member has its own dimensions.</p>

Calculate the mean, second moment, centred variance and SE using the current cell probabilities; return a list.

$$
\widehat{\operatorname{var}}(D)=\sum_jp_j(D_j-\overline D)^2,\quad\widehat{SE}=\sqrt{\widehat{\operatorname{var}}(D)/n}
$$


</div>

<div class="r-line-card" data-r-line="22">

#### R line 22

```r
saturated_values <- eif_values(data$empirical_model, data$cells)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions. Output: <code>saturated_values</code> · numeric vector · Length 8; aligned according to (x, a, y) grid.</p>

Evaluate eight EIF values under the saturated model of the full empirical distribution, distinct from the initial Q fitted using A alone.

</div>

<div class="r-line-card" data-r-line="24">

#### R line 24

```r
saturated_moments <- if_variance(data, saturated_values)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>saturated_values</code> · numeric vector · Length 8; aligned according to (x, a, y) grid. Output: <code>saturated_moments</code> · list · Named container; each member has its own dimensions.</p>

Use the same probability weights to obtain a second list of empirical moments.

</div>

<div class="r-line-card" data-r-line="26">

#### R line 26

```r
print(list(initial = initial_moments, saturated = saturated_moments))
```

<p>Input: <code>initial_moments</code> · list · Named container; each member has its own dimensions; <code>saturated_moments</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display both sets of empirical means, second moments, variances and SEs by name. These are not known efficiency bounds at P0.

</div>

<div class="r-line-card" data-r-line="28">

#### R line 28

```r
cat("initial SE in percentage points =", 100 * initial_moments$se, "; saturated =", 100 * saturated_moments$se, "\n")
```

<p>Input: <code>initial_moments</code> · list · Named container; each member has its own dimensions; <code>saturated_moments</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Multiply each standard error by 100 to convert a probability difference to percentage points. A variance would instead be multiplied by 10,000 to express it in squared percentage points.

</div>

</details>

</div>



<details><summary>When does an empirical standard error support valid inference?</summary>

The current workbench recalculates a given function’s average, second moment and centred variance. After updating $Q$ in Chapter 8, we must also recalculate $D$. Valid inference from these empirical calculations additionally requires checking convergence of fitted functions, remainder rates, sampling dependence and clinical identification conditions. This course retains the SE calculation exercise; the coverage of clinical confidence intervals still requires validation.

</details>

## Next: control estimation error using AIPW and TMLE

We have moved from the mean’s exact expansion to general asymptotic linearity, then to $V/n$, the CLT and efficiency lower bounds. The code calculates empirical moments for a given function. The next chapter connects these conditions to AIPW corrections and TMLE updates, checking numerical equations and remaining error.


---

# Chapter 8: Calculate AIPW and TMLE step by step in the RHC cohort

Now start from the original CSV and calculate the estimates. The preceding chapters established the target, EIF, directional checks, model constraints and connection to estimation error; this chapter constructs initial fits, bias corrections and risk updates in sequence. First calculate by hand using sex strata, then expand to the specified baseline variables for the same 5735 patients. We follow the same statistical target throughout the calculation; causal interpretation uses Chapter 0’s identification conditions.

## 1. Why does a first-order expansion suggest bias correction?

Let $P_0$ be the unknown true observed-data distribution, with $\psi_0=\Psi(P_0)$. Hats denote sample fits, and $P_nf=n^{-1}\sum_i f(O_i)$ is the empirical average. The initial plug-in estimate is:

$$
\widehat\psi_{\rm plug}=\frac1n\sum_i[\widehat Q(1,W_i)-\widehat Q(0,W_i)].
$$

> **Notation recap:** Hold each person’s $W$ fixed and predict under $A=1$ and $A=0$. Average both sets of predictions over the same $n$ people, retaining identical baseline composition. The function $Q$ is fitted mortality probability and has not yet been updated.

The distributional gradient suggests that, if the initial fit leaves residuals in the target direction, the fitted EIF’s sample average can provide a first-order correction. The centring term cancels exactly when averaged over the same empirical $W$, leaving:

$$
\widehat\psi_{\rm AIPW}=\widehat\psi_{\rm plug}
+\frac1n\sum_i\widehat H_i[Y_i-\widehat Q(A_i,W_i)],
\quad \widehat H_i=\frac{A_i}{\widehat g(W_i)}-\frac{1-A_i}{1-\widehat g(W_i)}.
$$

> **Notation recap:** AIPW means augmented inverse probability weighting, also called a one-step correction. The weight $H$ comes from Chapter 4’s quotient; $Y-Q$ is the residual under the observed treatment. It corrects the final number while leaving initial $Q$ unchanged.

## 2. What else does valid inference require beyond double robustness?

First treat the fitted functions as given. Let $e_a=\widehat Q_a-Q_{0a}$ and $\delta g=\widehat g-g_0$. Given $W$, the weighted residual has mean $-g_0e_1/\widehat g+(1-g_0)e_0/(1-\widehat g)$. The initial conditional difference exceeds the true difference by $e_1-e_0$. Adding and collecting coefficients gives:

$$
\begin{aligned}
B&=\mathbb E_0\left[e_1-e_0-\frac{g_0}{\widehat g}e_1+\frac{1-g_0}{1-\widehat g}e_0\right]\\
&=\mathbb E_0\left[\left(1-\frac{g_0}{\widehat g}\right)e_1+
\left(\frac{1-g_0}{1-\widehat g}-1\right)e_0\right]\\
&=\mathbb E_0\left[(\widehat g-g_0)
\left\{\frac{e_1}{\widehat g}+\frac{e_0}{1-\widehat g}\right\}\right].
\end{aligned}
$$

> **Notation recap:** $B$ is defined as the population-averaged AIPW expression minus the true target, involving unknown $P_0$. If $Q$ is correct in both arms, or $g$ is correct, the product is zero.

Double robustness follows from this global product identity. Neyman orthogonality describes a local derivative near the true functions: small nuisance-function errors have no first-order effect. Valid statistical inference additionally requires a remainder smaller than $n^{-1/2}$, a fitted influence function approaching the true EIF, and appropriate sampling and empirical-process conditions.

For example, if both error norms shrink only at rate $n^{-1/4}$, their product is $n^{-1/2}$, which is not negligible on the required scale. At least one error must shrink faster for this rate argument to establish the requirement. Cross-fitting helps control empirical-process terms caused by training dependence. Clinical confounding and original-cohort selection are addressed through study design and identification assumptions. This chapter’s main-effects regressions demonstrate the full calculation; the chosen fitting methods still require justification of their asymptotic conditions.

## 3. Why does TMLE change Q rather than directly adding a number?

TMLE retains the same target rule and updates initial risks in the direction identified by the EIF. Define logit (log odds) and its inverse expit:

$$
\operatorname{logit}(q)=\log\frac q{1-q},\quad
\operatorname{expit}(u)=\frac1{1+e^{-u}},\quad
Q_\epsilon(a,w)=\operatorname{expit}\left[\operatorname{logit}\widehat Q(a,w)+\epsilon H_{\widehat g}(a,w)\right].
$$

> **Notation recap:** The initial logit is a fixed offset with coefficient 1. The clever covariate $H$ comes from the EIF’s residual weight; $\epsilon$ is the only new parameter estimated in this update. For $a=0$, $H$ is negative; for $a=1$, it is positive. Both predictions must be updated.

During updating, $g$, the empirical $W$ distribution and original patient records stay fixed; only predicted risks change. Expit maps finite real numbers into the interval between 0 and 1. If an operation exceeds the representable numerical range, the programme reports a numerical boundary error.

## 4. Derive the logistic update score with the chain rule

The linear predictor for the observed treatment is $\eta_i=\operatorname{logit}\widehat Q(A_i,W_i)+\epsilon\widehat H_i$; write $q_i=\operatorname{expit}(\eta_i)$. The average log-likelihood for a binary outcome is:

$$
\ell_n(\epsilon)=\frac1n\sum_i[Y_i\log q_i+(1-Y_i)\log(1-q_i)].
$$

> **Notation recap:** $q_i$ selects only the prediction under each person’s observed treatment $A$; $Y=1$ uses $\log q$ and $Y=0$ uses $\log(1-q)$. The parameter $\epsilon$ changes $q$ through $\eta$.

First note $dq_i/d\eta_i=q_i(1-q_i)$ and $d\eta_i/d\epsilon=H_i$, then multiply through the layers:

$$
\begin{aligned}
\frac{d\ell_n}{d\epsilon}
&=\frac1n\sum_i\left[\frac{Y_i}{q_i}-\frac{1-Y_i}{1-q_i}\right]q_i(1-q_i)H_i\\
&=\frac1n\sum_i[Y_i(1-q_i)-(1-Y_i)q_i]H_i\\
&=\frac1n\sum_i[Y_i-Y_iq_i-q_i+Y_iq_i]H_i\\
&=\frac1n\sum_iH_i(Y_i-q_i)=s_n(\epsilon),\\
\frac{ds_n}{d\epsilon}&=-\frac1n\sum_iH_i^2q_i(1-q_i)<0.
\end{aligned}
$$

> **Notation recap:** $s_n$ is the function in the likelihood score equation, giving the likelihood’s slope in $\epsilon$. The middle terms $-Y_iq_i+Y_iq_i$ cancel, leaving a residual. The negative slope ensures uniqueness whenever a finite root exists; that root also maximises the likelihood along this one-dimensional path.

If $s_n(0)>0$, increase $\epsilon$ to lower the score. Bisection first finds endpoints with opposite signs and tests their midpoint. A positive score retains the interval to the midpoint’s right; a negative score retains the interval to its left. Both the sex-stratified and multivariable experiments permit step-by-step manual operation; automatic root-finding runs only after you click.

## 5. First calculate TMLE manually with real grouped counts

The four groups’ offsets are the logits of their observed treatment arms’ initial mortality proportions; $g$ is estimated from within-sex counts. Each group summarises its score using its patient count $n_{xa}$ and death count $m_{xa}$:

$$
s_n(\epsilon)=\frac1{5735}\sum_{x,a}H(a,x)[m_{xa}-n_{xa}Q_\epsilon(a,x)].
$$

> **Notation recap:** $m$ is the observed death count and $n_{xa}Q$ the predicted death count. Subtract and multiply by $H$; dividing by 5735 gives a per-record average.

Predict the score’s sign, enter $\epsilon$, and perform at least five manual bisections before opening the answer. Once the root is found, standardise again: $\widehat\psi_T=\sum_x\widehat p_X(x)[Q^*(1,x)-Q^*(0,x)]$, where $Q^*=Q_{\widehat\epsilon}$.

<div class="case-workbench" data-case-study="8"></div>

<details><summary>Why do sex-stratified AIPW and TMLE give the same result?</summary>

The sex-stratified result is approximately 7.369212 percentage points. Because $g$ is saturated using the treatment proportion observed within each sex stratum, for any $Q$ the average weighted residual is exactly “saturated sex-standardised difference minus current plug-in difference”. Solving the score equation therefore makes the updated plug-in equal the saturated standardised difference; AIPW’s one-step correction gives the same value. Their equality here follows from a count identity with saturated $g$ and applies to this sex-stratified construction. Clinical interpretation of the sex-standardised result still uses Chapter 0’s identification conditions.

</details>

## 6. Expand the baseline adjustment and fit models for the same patients

Now expand the single sex variable $X$ to baseline vector $W$, keeping the same data and outcomes. Use the original age, sex, race and 12 medical-history variables. These columns are complete, so no patient removal or imputation is needed. Standardise age; use white as the reference race category, with black/other indicator columns. Race retains the social classifications recorded in the original data. There are 17 columns including the intercept. Initial $g$ uses these columns to fit RHC; $Q$ adds observed $A$ to fit mortality.

We choose main-effects logistic models for the initial fits. Iterations use the likelihood, gradient and information matrix, checking convergence, design-matrix rank and probability range. Clinical justification of the chosen baseline variables and timing of acute severity measurements must be assessed separately under Chapter 0’s study protocol.

### Calculate risk updates, weighted residuals and risk differences column by column

The experiment below always retains the same original row $i$ and observed $A/Y$. Read the current step and predict, then click “Next step” to add columns on the right; existing columns remain. The sequence is: three initial predictions → $g$ and $H$ → observed-arm offset → $\epsilon H$ and new logit → updated probability → residual and score contribution → separate updates for all $A=0$ and all $A=1$ → common-population average → full EIF. The table shows only six rows for checking; all fitting, root-finding and averaging use all 5735 patients.

In each step’s linked formula, symbol buttons correspond to **the relevant code variable** and **the same data column**. For example, $\epsilon H_{A,i}$ corresponds to `shift`, the initial logit to `offset`, and updated probability to `qstar_a`. Scalars $\epsilon$, $n$ and the risk difference each have a parameter or summary card.

> **Notation recap:** The offset stays fixed at the logit of the initial prediction; $H$ comes from initial $g$ and the specified treatment level, and $\epsilon$ is one new parameter shared by all patients. Observed-arm updates estimate $\epsilon$; both-arm updates supply final standardisation. During trials, code variable `qstar_a` denotes the current $Q_\epsilon$; only after root-finding is the corresponding update called $Q^*$.

<div class="rhc-full" data-rhc-full></div>

Advance to “Residual and score contribution” and inspect the initial score. Enter 0.003 and 0.004 manually, record their signs, then bisect. After updating $Q$, calculate updated predictions under both treatments for **every patient**, obtain 5735 differences, and average the entire column.

<details><summary>Check the estimates after expanding baseline variables</summary>

Under the specified models, initial plug-in is approximately 8.649886 percentage points and AIPW 8.996800; $\epsilon$ is about 0.003576966663, TMLE about 8.997972 percentage points, and average score about −3.4×10⁻¹³. The $g$ range is approximately 0.078698–0.681905, with no truncation. The webpage, Julia and R implementations cross-check the unrounded results independently.

These numbers estimate the specified statistical functional. Their differences from the sex-stratified result reflect the changed adjustment set and initial models. Clinical causal interpretation additionally requires justification of sufficient adjustment and Chapter 0’s study-design assumptions.

</details>

## 7. Why does the empirical EIF equation become zero? What errors remain?

The plug-in definition ensures $P_n[Q^*(1,W)-Q^*(0,W)-\widehat\psi_T]=0$; targeting ensures $P_n[H(Y-Q^*)]\approx0$. Adding gives an approximately zero sample average of the full fitted EIF; individual records retain their residuals.

Using the bias identity for the fitted functions:

$$
\widehat\psi_T-\psi_0=(P_n-P_0)\widehat D^*+B(Q^*,\widehat g)-P_n\widehat D^*.
$$

> **Notation recap:** The first term is the difference between empirical and population averages; the second remains the preceding error product, now with updated $Q$. The last term is zero and may be omitted only for an exact solution; numerical solutions retain their root-finding tolerance. The second term’s size depends jointly on the fitting errors of $Q,g$. Replacing the first term by $(P_n-P_0)D_0^*$ also requires controlling the empirical process of the difference between fitted and true functions.

Recalculate updated empirical variance and SE as in Chapter 7. Valid clinical confidence intervals additionally require the asymptotic conditions above and Chapter 0’s identification conditions; finite-sample errors of TMLE and AIPW also need separate comparison. Interpret the first-day RHC window, 48-hour selection and treatment versions under Chapter 0’s protocol.

## Recalculate initial fits, risk updates and bisection in code

<div class="lesson-language-control"><label>Code language in this chapter <select data-lesson-language><option value="julia">Julia operations</option><option value="r">R operations</option></select></label><span>Use the same RHC data and the same mathematical objectives</span></div>

<div class="lesson-code-panel" data-lesson-panel="julia">


The [column-by-column workbook](julia/tmle_trace.jl) follows the same table above, constructing every intermediate column line by line from the original CSV. Its default is $\epsilon=0$; an environment variable can supply a trial value, and the script does not find the root for you automatically. “Julia for this step” and “Full column-by-column workbook” below the figure read actual source code and explain inputs, outputs, dimensions, broadcasting and mathematical expressions line by line.

```sh
julia --startup-file=no --history-file=no en/julia/tmle_trace.jl
TMLE_EPSILON=0.003 julia --startup-file=no --history-file=no en/julia/tmle_trace.jl
```

`TMLE_EPSILON` arrives as a string; `parse(Float64, ...)` converts it to a number, defaulting to 0 when unset. The operators `.*`, `./`, `.-` multiply, divide and subtract elementwise. For example, `offset .+ shift` still outputs an $n$-element vector; only `sum(contrast) / n` reduces it to a risk-difference scalar. The call `expit.(eta)` converts each updated logit back to probability; the earlier `offset` comes from the logit of predicted probability `qa`. Displayed decimals are rounded; workbooks and assertions use unrounded values.

The original Chapter 8 script still calculates within sex strata first, then runs the multivariable version on the same original CSV. It outputs input dimensions, the $g$ range, initial score, midpoint/score for the first five bisections, and final $\epsilon$ and updated risk difference. The webpage additionally lists the offset, $H$ and retained interval; the full module stores the corresponding calculations. Final automatic checks use the same equation without calling a TMLE package. In the [full module](../shared/julia/RHCCase.jl), `BEGIN_FULL_BASELINE` and `BEGIN_FULL_TARGET` retain the actual fitting and update source code.

The input is [the same original RHC CSV](../shared/data/public/rhc/source.csv). For reading and fitting definitions, see [RHCCase.jl](../shared/julia/RHCCase.jl). The complete [chapter script](julia/rhc_lessons/ch08.jl) is editable; all scripts can be run from other working directories, as long as the relative positions of en/ and shared/ are retained.

```sh
julia --startup-file=no --history-file=no en/julia/rhc_lessons/ch08.jl
```

```julia
# Public RHC cohort: X=sex, A=RHC receipt, Y=30-day death; all quantities are computed from original records.
# BEGIN_CH08 defines expit, score, four-group updates, bisection and failure checks.
# This script explicitly tries epsilon values and progressively narrows the bracket; no TMLE package is called.
include(joinpath(@__DIR__, "..", "..", "..", "shared", "julia", "RHCCase.jl"))
using .RHCCase
using Printf

function manual_tmle(data; lo=-1.0, hi=1.0, tolerance=1e-12, maxiter=200)
    # Each score is the group sum of H×(observed events−predicted events), divided by total patient count.
    for epsilon in (0.0, 0.1, 0.2)
        @printf("Trial epsilon=%.3f: score=%+.9f\n", epsilon, fluctuation_score(data, epsilon).score)
    end
    fluctuation_score(data, lo).score >= 0 && fluctuation_score(data, hi).score <= 0 ||
        error("Initial bracket does not contain the score root")
    for iteration in 1:maxiter
        step = bisection_step(data, lo, hi)
        if iteration <= 5
            @printf("Bisection %d: epsilon=%.9f, score=%+.9f\n", iteration, step.midpoint, step.score)
        end
        if abs(step.score) <= tolerance
            # case_at computes expit(offset+epsilon×H) in each group, then averages over empirical X proportions.
            result = case_at(data, step.midpoint)
            for group in result.groups
                @printf("x=%d,a=%d: Qstar=%.12f, Score contribution=%+.12f\n", group.x, group.a, group.q_updated, group.score_contribution)
            end
            @printf("epsilon=%.12f, Sex-standardised risk difference=%.12f, Empirical EIF mean=%.3g, SE=%.9f\n", result.epsilon, result.estimate, result.eif_mean, result.se)
            return result
        end
        step.midpoint != lo && step.midpoint != hi || error("Bisection stopped at floating-point precision")
        lo, hi = step.lo, step.hi
    end
    error("Maximum iterations reached before finding a score root within tolerance")
end

data = rhc_data()
manual_tmle(data)

println("Standardisation here uses sex alone, without establishing sufficient confounding control; results are not clinical causal effects of RHC.")

# For the same 5735 patients, finally expand X from sex to the prespecified 17-column baseline design.
# See BEGIN_FULL_BASELINE in RHCCase.jl for logistic-regression Newton steps and likelihood backtracking.
# See BEGIN_FULL_TARGET for patient-level updates and score; no TMLE or regression packages are used.
function manual_baseline_tmle(fit; lo=-0.25, hi=0.25, tolerance=1e-12, maxiter=200)
    println("Expanded baseline columns = ", fit.feature_names)
    println("Age standardisation: mean=", fit.age_mean, ", Population SD=", fit.age_sd)
    println("g minimum/maximum=", extrema(fit.g), "; No truncation or imputation")
    initial = full_at(fit, 0.0)
    @printf("Expanded-baseline initial risk difference=%+.9f; AIPW=%+.9f; score=%+.9f\n", initial.estimate, initial.aipw, initial.score)
    for epsilon in (0.0, 0.002, 0.004)
        state = full_at(fit, epsilon)
        @printf("Patient-level trial epsilon=%.3f: score=%+.9f, Mean log likelihood=%.9f\n", epsilon, state.score, state.log_likelihood)
    end
    for iteration in 1:maxiter
        step = full_bisection_step(fit, lo, hi)
        if iteration <= 5
            @printf("Expanded-baseline bisection %d: epsilon=%.9f, score=%+.9f\n", iteration, step.midpoint, step.score)
        end
        if abs(step.score) <= tolerance
            state = full_at(fit, step.midpoint)
            @printf("Expanded-baseline epsilon=%.12f: risk0=%.9f, risk1=%.9f\n", state.epsilon, state.risk0, state.risk1)
            @printf("TMLE risk difference=%+.9f; score=%.3g; Empirical EIF mean=%.3g; Empirical SE=%.9f\n", state.estimate, state.score, state.eif_mean, state.se)
            return state
        end
        step.midpoint != lo && step.midpoint != hi || error("Expanded-baseline bisection stopped at floating-point precision")
        lo, hi = step.lo, step.hi
    end
    error("Expanded-baseline bisection did not converge within the iteration limit")
end

full_fit = prepare_full(read_rhc_rows())
manual_baseline_tmle(full_fit)
println("The expanded baseline model still omits acute illness severity; the data do not establish sufficient confounding control or the selection assumptions.")
println("Empirical SE reflects the current model’s asymptotic derivation; it does not guarantee confidence-interval coverage or clinical causal identification.")
```


<details><summary>View the source for this chapter’s mathematical functions</summary>

RHCCase reuses the following functions with counts from the source data. They contain no built-in values for the unknown population target.

```julia
function expit(z::Real)
    isfinite(z) || throw(ArgumentError("Finite log odds required"))
    z >= 0 ? 1 / (1 + exp(-z)) : exp(z) / (1 + exp(z))
end
function logit(q::Real)
    isfinite(q) && 0 < q < 1 || throw(ArgumentError("Probability must be strictly interior"))
    log(q) - log1p(-q)
end
softplus(z) = max(z, zero(z)) + log1p(exp(-abs(z)))

# H and offset are added during updating; Chapter 3 count summaries do not depend on them.
function fluctuation_groups(data)
    [merge(group, (h=clever_h(data.initial_fit, group.a, group.x),
                   offset=logit(group.initial_q))) for group in data.groups]
end

# The mean conditional log-likelihood, its score and its derivative depend only on observed group counts.
function fluctuation_score(data, epsilon::Real)
    isfinite(epsilon) || throw(ArgumentError("Finite fluctuation parameter required"))
    score = 0.0; slope = 0.0; log_likelihood = 0.0
    for group in fluctuation_groups(data)
        eta = group.offset + epsilon * group.h
        isfinite(eta) || throw(ArgumentError("Updated log odds are not representable"))
        q = expit(eta)
        score += group.h * (group.events - group.n * q) / data.n
        slope -= group.n * group.h^2 * q * (1 - q) / data.n
        log_likelihood -= (group.events * softplus(-eta) +
                           (group.n - group.events) * softplus(eta)) / data.n
    end
    all(isfinite, (score, slope, log_likelihood)) ||
        throw(ArgumentError("Fluctuation summary is not representable"))
    (; score, slope, log_likelihood)
end

function case_at(data, epsilon::Real=0.0)
    state = fluctuation_score(data, epsilon)
    q = similar(data.initial_fit.q)
    groups = map(fluctuation_groups(data)) do group
        updated_q = expit(group.offset + epsilon * group.h)
        0 < updated_q < 1 || throw(ArgumentError("Updated Q reached a floating-point endpoint"))
        q[group.a+1,group.x+1] = updated_q
        predicted_events = group.n * updated_q
        residual_total = group.events - predicted_events
        weighted_residual = group.h * residual_total
        merge(group, (; q_updated=updated_q, predicted_events, residual_total,
                       weighted_residual, score_contribution=weighted_residual / data.n))
    end
    updated_fit = (px=copy(data.initial_fit.px), g=copy(data.initial_fit.g), q=q)
    estimate = ate(updated_fit)
    values = eif_values(updated_fit, data.cells; estimate)
    moments = if_variance(data, values)
    eif_rows = [merge(c, (d=values[i],)) for (i,c) in enumerate(data.cells)]
    (; data, epsilon=Float64(epsilon), groups, state.score, state.slope, state.log_likelihood,
       updated_fit, estimate, aipw=estimate + state.score, eif_rows,
       eif_mean=moments.mean, eif_second_moment=moments.second_moment,
       eif_variance=moments.variance, se=moments.se)
end

function checked_bracket(data, lo, hi)
    isfinite(lo) && isfinite(hi) && lo < hi || throw(ArgumentError("Finite increasing bracket required"))
    slo = fluctuation_score(data, lo).score
    shi = fluctuation_score(data, hi).score
    slo >= 0 && shi <= 0 || throw(ArgumentError("Bracket must contain the decreasing score root"))
    (; lo=Float64(lo), hi=Float64(hi), slo, shi)
end

function bisection_step(data, lo::Real, hi::Real)
    bracket = checked_bracket(data, lo, hi)
    midpoint = bracket.lo / 2 + bracket.hi / 2
    score = fluctuation_score(data, midpoint).score
    new_lo = score > 0 ? midpoint : bracket.lo
    new_hi = score < 0 ? midpoint : bracket.hi
    score == 0 && (new_lo = new_hi = midpoint)
    (; midpoint, score, lo=new_lo, hi=new_hi)
end

function solve_case(data; lo::Real=-1.0, hi::Real=1.0, score_tol::Real=1e-12, maxiter::Integer=200)
    isfinite(score_tol) && 0 < score_tol <= 1e-6 || throw(ArgumentError("Use a positive score tolerance at most 1e-6"))
    maxiter > 0 || throw(ArgumentError("Positive maximum iteration count required"))
    bracket = checked_bracket(data, lo, hi)
    lo, hi = bracket.lo, bracket.hi
    for (epsilon, score) in ((lo, bracket.slo), (hi, bracket.shi))
        if abs(score) <= score_tol
            return merge(case_at(data, epsilon), (converged=true, iterations=0, reason="score_tolerance"))
        end
    end
    for iterations in 1:maxiter
        step = bisection_step(data, lo, hi)
        if abs(step.score) <= score_tol
            return merge(case_at(data, step.midpoint), (; converged=true, iterations, reason="score_tolerance"))
        end
        if step.midpoint == lo || step.midpoint == hi
            return merge(case_at(data, step.midpoint), (; converged=false, iterations, reason="floating_point_stagnation"))
        end
        lo, hi = step.lo, step.hi
    end
    merge(case_at(data, lo / 2 + hi / 2), (converged=false, iterations=Int(maxiter), reason="maximum_iterations"))
end
```

</details>



</div>

<div class="lesson-code-panel" data-lesson-panel="r" hidden>

The input is [the same original public RHC CSV](../shared/data/public/rhc/source.csv). Shared definitions: [RHCCase.R](../shared/R/RHCCase.R) and [EIFCase.R](../shared/R/EIFCase.R). [Complete R script](R/ch08.R) uses base R and runs from any working directory if you retain the project directory structure.

```sh
Rscript --vanilla en/R/ch08.R
```

```r
# Chapter 8: AIPW and explicit manual bisection for TMLE
# Input: frozen original public RHC CSV; output: chapter objects and terminal checks; base R only.
# commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
# sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
# if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch08.R")
# length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.
stopifnot(length(script_path) == 1L, !is.na(script_path))
# dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
# source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.
source(file.path(project_root, "shared", "R", "RHCCase.R"))
# Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.
data <- rhc_data()
# Manually try three epsilon scalars in sequence. The for loop only evaluates the score; it does not solve for a root.
for (epsilon in c(0, 0.1, 0.2)) {
# Calculate one empirical score for each trial coefficient using the fixed initial offset and H.
  cat("trial epsilon =", epsilon, "; score =", fluctuation_score(data, epsilon)$score, "\n")
# End the manual trial loop; the original data and fitted models remain fixed.
}
# Set the lower endpoint of the root bracket for the sex-stratified calculation.
lo <- -1
# Set the upper endpoint of the root bracket for the sex-stratified calculation.
hi <- 1
# Set the tolerance for the score’s absolute value. Approximate zero remains distinct from exact zero.
tolerance <- 1e-12
# Check that the non-increasing score brackets a root between the interval endpoints; stop if it does not.
stopifnot(fluctuation_score(data, lo)$score >= 0, fluctuation_score(data, hi)$score <= 0)
# NULL marks a root result that has not yet been calculated. Store the updated object after a successful solve.
targeted <- NULL
# Run at most 200 explicit bisection steps, calculating one interval midpoint each time.
for (iteration in 1:200) {
# The shared pure function returns midpoint, score and the narrower lo/hi bracket.
  step <- bisection_step(data, lo, hi)
# Display the first five bisection steps for checking; all iterations follow the same rule.
  if (iteration <= 5) cat("bisection", iteration, "; epsilon =", step$midpoint, "; score =", step$score, "\n")
# abs takes the score’s absolute value. Enter the success branch only when it meets the specified tolerance.
  if (abs(step$score) <= tolerance) {
# Update the four Q values using the solved midpoint, standardise using empirical X proportions, and calculate the complete empirical EIF and SE.
    targeted <- case_at(data, step$midpoint)
# break exits the innermost for loop, retaining the successful targeted object.
    break
# End the convergence-check branch.
  }
# Stop when floating-point precision makes the midpoint equal an endpoint, so the interval cannot be narrowed further.
  stopifnot(step$midpoint != lo, step$midpoint != hi)
# Replace the lower bound with the new bound selected by the sign of the score.
  lo <- step$lo
# Replace the upper bound with the new bound selected by the sign of the score.
  hi <- step$hi
# End the bisection loop. targeted remains NULL if convergence has not been reached.
}
# Report failure explicitly if the iteration limit is reached without a result meeting tolerance.
if (is.null(targeted)) stop("Eight-cell bisection did not converge")
# Display the four updated Q values and score contributions. Standardise both arms using the original empirical X weights.
print(targeted$groups)
# Display the sex-stratified TMLE risk difference, empirical EIF mean within tolerance, and SE. Adjustment here uses sex only.
cat("eight-cell epsilon =", targeted$epsilon, "; estimate =", targeted$estimate, "; EIF mean =", targeted$eif_mean, "; SE =", targeted$se, "\n")
# Reread the same 5,735 original patient records. Expanding the adjustment set does not change the cohort.
rows <- read_rhc_rows(DEFAULT_SOURCE)
# Construct the baseline design in a fixed column order: n×17 for g and n×18 for Q. Age scaling uses denominator n, white is the race reference, and there is no regularisation or implicit truncation.
fit <- prepare_full(rows)
# epsilon=0 retains the initial prediction and returns the initial standardised risk difference, AIPW and empirical score.
initial <- full_at(fit, 0)
# Display the design columns and age-standardisation parameters so readers can check the fixed adjustment set.
cat("baseline columns =", fit$feature_names, "; age mean =", fit$age_mean, "; age SD =", fit$age_sd, "\n")
# range displays the minimum and maximum across all n fitted g values. Keep the initial plug-in estimate, AIPW and score distinct.
cat("g range =", range(fit$g), "; initial =", initial$estimate, "; AIPW =", initial$aipw, "; score =", initial$score, "\n")
# Manually try three update coefficients with the full baseline model, then find the root by bisection.
for (epsilon in c(0, 0.002, 0.004)) {
# For each patient, calculate the update using fixed offset plus epsilon×H, then calculate the score and mean log likelihood.
  trial <- full_at(fit, epsilon)
# Display the mean score and mean log likelihood over all 5,735 patients; the denominator is the full cohort size.
  cat("full trial epsilon =", epsilon, "; score =", trial$score, "; log likelihood =", trial$log_likelihood, "\n")
# End the manual trials for the full baseline model.
}
# Set the lower bisection bound for the full baseline model’s score.
lo <- -0.25
# Set the upper bisection bound for the full baseline model’s score.
hi <- 0.25
# Check that the non-increasing score for the full baseline model brackets a root. Stop explicitly if this fails.
stopifnot(full_at(fit, lo)$score >= 0, full_at(fit, hi)$score <= 0)
# Use NULL to mark that no result for the full baseline model has yet met tolerance.
full_targeted <- NULL
# Start up to 200 explicit bisection steps for the full baseline model.
for (iteration in 1:200) {
# Evaluate the midpoint score and narrow the bracket using the same fit, keeping all initial Q and g values fixed.
  step <- full_bisection_step(fit, lo, hi)
# Display the midpoint and score for the first five patient-level bisection steps so readers can check them manually.
  if (iteration <= 5) cat("full bisection", iteration, "; epsilon =", step$midpoint, "; score =", step$score, "\n")
# Check the full n-patient score against the same prespecified tolerance, 1e−12.
  if (abs(step$score) <= tolerance) {
# Name the result targeted only after solving for the root. Both arms use the same epsilon and are standardised over the same n patients.
    full_targeted <- full_at(fit, step$midpoint)
# Exit the bisection loop for the full baseline model.
    break
# End the convergence branch for the full baseline model.
  }
# Stop immediately when floating-point precision prevents further narrowing, avoiding false convergence.
  stopifnot(step$midpoint != lo, step$midpoint != hi)
# Keep the new lower bound of the bracket containing the root.
  lo <- step$lo
# Keep the new upper bound of the bracket containing the root.
  hi <- step$hi
# End the full baseline model’s bisection loop.
}
# Report an error when iterations are exhausted; success is defined by the score tolerance.
if (is.null(full_targeted)) stop("Full baseline bisection did not converge")
# Display the solved coefficient, each arm’s mean risk over all n patients, and the standardised risk difference.
cat("full epsilon =", full_targeted$epsilon, "; risk0 =", full_targeted$risk0, "; risk1 =", full_targeted$risk1, "; TMLE =", full_targeted$estimate, "\n")
# Display the likelihood score, full EIF mean and empirical SE separately. See this chapter’s technical notes for finite-sample and inference conditions.
cat("full score =", full_targeted$score, "; EIF mean =", full_targeted$eif_mean, "; empirical SE =", full_targeted$se, "\n")
# The full baseline still omits acute illness severity; data do not establish sufficient confounding control or the selection assumptions.
```

<details><summary>Line-by-line explanation: input, output, dimensions and mathematical expressions</summary>

<div class="r-line-card" data-r-line="4">

#### R line 4

```r
script_argument <- grep("^--file=", commandArgs(FALSE), value = TRUE)
```

<p>Input: constants or the current R environment. Output: <code>script_argument</code> · character vector · path or parameter string.</p>

commandArgs(FALSE) reads the startup arguments. grep matches the start of a string using ^, and value=TRUE retains the matched strings. The result is a character vector of length zero or one locating the file run by Rscript.

</div>

<div class="r-line-card" data-r-line="6">

#### R line 6

```r
source_files <- vapply(sys.frames(), function(frame) if (is.null(frame$ofile)) "" else frame$ofile, character(1))
```

<p>Input: constants or the current R environment. Output: <code>source_files</code> · character vector · path or parameter string.</p>

sys.frames() reads the call frames. vapply applies a function to each frame, requiring one character string per result. When source() runs a script, ofile records its path; frames without a path return an empty string.

</div>

<div class="r-line-card" data-r-line="8">

#### R line 8

```r
script_path <- if (any(nzchar(source_files))) tail(source_files[nzchar(source_files)], 1) else if (length(script_argument)) sub("^--file=", "", script_argument[1]) else file.path("en", "R", "ch08.R")
```

<p>Input: <code>source_files</code> · character vector · path or parameter string; <code>script_argument</code> · character vector · path or parameter string. Output: <code>script_path</code> · character vector · path or parameter string.</p>

if selects the source path: use the last non-empty ofile when called through source(); otherwise remove the --file= prefix. Code pasted into the R console at the project root defaults to this lesson’s script path. The output is one path string; [] indexing starts at 1.

</div>

<div class="r-line-card" data-r-line="10">

#### R line 10

```r
stopifnot(length(script_path) == 1L, !is.na(script_path))
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

length checks that the path has one element. 1L is integer 1, and !is.na excludes missing values. stopifnot requires every condition to be TRUE and stops otherwise.

</div>

<div class="r-line-card" data-r-line="12">

#### R line 12

```r
project_root <- normalizePath(file.path(dirname(script_path), "..", ".."), mustWork = TRUE)
```

<p>Input: <code>script_path</code> · character vector · path or parameter string. Output: <code>project_root</code> · character vector · path or parameter string.</p>

dirname extracts the script directory. file.path joins the path two levels up to the project root; normalizePath requires that directory to exist. The result is independent of the current working directory.

</div>

<div class="r-line-card" data-r-line="14">

#### R line 14

```r
source(file.path(project_root, "shared", "R", "RHCCase.R"))
```

<p>Input: <code>project_root</code> · character vector · path or parameter string. Output: checks, displayed output or control flow.</p>

source reads and executes the shared R definitions for reading the CSV, mathematical functions and manual model fitting. Only base R is used; no TMLE package is required.

</div>

<div class="r-line-card" data-r-line="16">

#### R line 16

```r
data <- rhc_data()
```

<p>Input: constants or the current R environment. Output: <code>data</code> · list · Named container; each member has its own dimensions.</p>

Read the original CSV, calculate empirical probabilities for the eight (x,a,y) cells and return the resulting data and model in a list. Later calculations use these same patient records.

</div>

<div class="r-line-card" data-r-line="18">

#### R line 18

```r
for (epsilon in c(0, 0.1, 0.2)) {
```

<p>Input: constants or the current R environment. Output: <code>epsilon</code> · numeric scalar (double) · Length 1.</p>

Manually try three epsilon scalars in sequence. The for loop only evaluates the score; it does not solve for a root.

</div>

<div class="r-line-card" data-r-line="20">

#### R line 20

```r
  cat("trial epsilon =", epsilon, "; score =", fluctuation_score(data, epsilon)$score, "\n")
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>epsilon</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Calculate one empirical score for each trial coefficient using the fixed initial offset and H.

$$
s_n(\epsilon)=\sum_jp_jH_j(y_j-Q_{\epsilon,j})
$$


</div>

<div class="r-line-card" data-r-line="22">

#### R line 22

```r
}
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

End the manual trial loop; the original data and fitted models remain fixed.

</div>

<div class="r-line-card" data-r-line="24">

#### R line 24

```r
lo <- -1
```

<p>Input: constants or the current R environment. Output: <code>lo</code> · numeric scalar (double) · Length 1.</p>

Set the lower endpoint of the root bracket for the sex-stratified calculation.

</div>

<div class="r-line-card" data-r-line="26">

#### R line 26

```r
hi <- 1
```

<p>Input: constants or the current R environment. Output: <code>hi</code> · numeric scalar (double) · Length 1.</p>

Set the upper endpoint of the root bracket for the sex-stratified calculation.

</div>

<div class="r-line-card" data-r-line="28">

#### R line 28

```r
tolerance <- 1e-12
```

<p>Input: constants or the current R environment. Output: <code>tolerance</code> · numeric scalar (double) · Length 1.</p>

Set the tolerance for the score’s absolute value. Approximate zero remains distinct from exact zero.

</div>

<div class="r-line-card" data-r-line="30">

#### R line 30

```r
stopifnot(fluctuation_score(data, lo)$score >= 0, fluctuation_score(data, hi)$score <= 0)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>lo</code> · numeric scalar (double) · Length 1; <code>hi</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Check that the non-increasing score brackets a root between the interval endpoints; stop if it does not.

</div>

<div class="r-line-card" data-r-line="32">

#### R line 32

```r
targeted <- NULL
```

<p>Input: constants or the current R environment. Output: <code>targeted</code> · list · Named container; each member has its own dimensions.</p>

NULL marks a root result that has not yet been calculated. Store the updated object after a successful solve.

</div>

<div class="r-line-card" data-r-line="34">

#### R line 34

```r
for (iteration in 1:200) {
```

<p>Input: constants or the current R environment. Output: <code>iteration</code> · integer scalar · Length 1.</p>

Run at most 200 explicit bisection steps, calculating one interval midpoint each time.

</div>

<div class="r-line-card" data-r-line="36">

#### R line 36

```r
  step <- bisection_step(data, lo, hi)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>lo</code> · numeric scalar (double) · Length 1; <code>hi</code> · numeric scalar (double) · Length 1. Output: <code>step</code> · list · Named container; each member has its own dimensions.</p>

The shared pure function returns midpoint, score and the narrower lo/hi bracket.

</div>

<div class="r-line-card" data-r-line="38">

#### R line 38

```r
  if (iteration <= 5) cat("bisection", iteration, "; epsilon =", step$midpoint, "; score =", step$score, "\n")
```

<p>Input: <code>iteration</code> · integer scalar · Length 1; <code>step</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the first five bisection steps for checking; all iterations follow the same rule.

</div>

<div class="r-line-card" data-r-line="40">

#### R line 40

```r
  if (abs(step$score) <= tolerance) {
```

<p>Input: <code>step</code> · list · Named container; each member has its own dimensions; <code>tolerance</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

abs takes the score’s absolute value. Enter the success branch only when it meets the specified tolerance.

</div>

<div class="r-line-card" data-r-line="42">

#### R line 42

```r
    targeted <- case_at(data, step$midpoint)
```

<p>Input: <code>data</code> · list · Named container; each member has its own dimensions; <code>step</code> · list · Named container; each member has its own dimensions. Output: <code>targeted</code> · list · Named container; each member has its own dimensions.</p>

Update the four Q values using the solved midpoint, standardise using empirical X proportions, and calculate the complete empirical EIF and SE.

$$
Q^*_{a,x}=\operatorname{expit}(\operatorname{logit}\widehat Q_{a,x}+\widehat\epsilon H_{a,x})
$$


</div>

<div class="r-line-card" data-r-line="44">

#### R line 44

```r
    break
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

break exits the innermost for loop, retaining the successful targeted object.

</div>

<div class="r-line-card" data-r-line="46">

#### R line 46

```r
  }
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

End the convergence-check branch.

</div>

<div class="r-line-card" data-r-line="48">

#### R line 48

```r
  stopifnot(step$midpoint != lo, step$midpoint != hi)
```

<p>Input: <code>step</code> · list · Named container; each member has its own dimensions; <code>lo</code> · numeric scalar (double) · Length 1; <code>hi</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Stop when floating-point precision makes the midpoint equal an endpoint, so the interval cannot be narrowed further.

</div>

<div class="r-line-card" data-r-line="50">

#### R line 50

```r
  lo <- step$lo
```

<p>Input: <code>step</code> · list · Named container; each member has its own dimensions. Output: <code>lo</code> · numeric scalar (double) · Length 1.</p>

Replace the lower bound with the new bound selected by the sign of the score.

</div>

<div class="r-line-card" data-r-line="52">

#### R line 52

```r
  hi <- step$hi
```

<p>Input: <code>step</code> · list · Named container; each member has its own dimensions. Output: <code>hi</code> · numeric scalar (double) · Length 1.</p>

Replace the upper bound with the new bound selected by the sign of the score.

</div>

<div class="r-line-card" data-r-line="54">

#### R line 54

```r
}
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

End the bisection loop. targeted remains NULL if convergence has not been reached.

</div>

<div class="r-line-card" data-r-line="56">

#### R line 56

```r
if (is.null(targeted)) stop("Eight-cell bisection did not converge")
```

<p>Input: <code>targeted</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Report failure explicitly if the iteration limit is reached without a result meeting tolerance.

</div>

<div class="r-line-card" data-r-line="58">

#### R line 58

```r
print(targeted$groups)
```

<p>Input: <code>targeted</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the four updated Q values and score contributions. Standardise both arms using the original empirical X weights.

</div>

<div class="r-line-card" data-r-line="60">

#### R line 60

```r
cat("eight-cell epsilon =", targeted$epsilon, "; estimate =", targeted$estimate, "; EIF mean =", targeted$eif_mean, "; SE =", targeted$se, "\n")
```

<p>Input: <code>targeted</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the sex-stratified TMLE risk difference, empirical EIF mean within tolerance, and SE. Adjustment here uses sex only.

</div>

<div class="r-line-card" data-r-line="62">

#### R line 62

```r
rows <- read_rhc_rows(DEFAULT_SOURCE)
```

<p>Input: constants or the current R environment. Output: <code>rows</code> · data.frame · 5735 rows × 63 columns.</p>

Reread the same 5,735 original patient records. Expanding the adjustment set does not change the cohort.

</div>

<div class="r-line-card" data-r-line="64">

#### R line 64

```r
fit <- prepare_full(rows)
```

<p>Input: <code>rows</code> · data.frame · 5735 rows × 63 columns. Output: <code>fit</code> · list · Named container; each member has its own dimensions.</p>

Construct the baseline design in a fixed column order: n×17 for g and n×18 for Q. Age scaling uses denominator n, white is the race reference, and there is no regularisation or implicit truncation.

</div>

<div class="r-line-card" data-r-line="66">

#### R line 66

```r
initial <- full_at(fit, 0)
```

<p>Input: <code>fit</code> · list · Named container; each member has its own dimensions. Output: <code>initial</code> · list · Named container; each member has its own dimensions.</p>

epsilon=0 retains the initial prediction and returns the initial standardised risk difference, AIPW and empirical score.

$$
\widehat\psi_{\rm AIPW}=\widehat\psi_{\rm init}+P_n\widehat D_{\rm init}
$$


</div>

<div class="r-line-card" data-r-line="68">

#### R line 68

```r
cat("baseline columns =", fit$feature_names, "; age mean =", fit$age_mean, "; age SD =", fit$age_sd, "\n")
```

<p>Input: <code>fit</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the design columns and age-standardisation parameters so readers can check the fixed adjustment set.

</div>

<div class="r-line-card" data-r-line="70">

#### R line 70

```r
cat("g range =", range(fit$g), "; initial =", initial$estimate, "; AIPW =", initial$aipw, "; score =", initial$score, "\n")
```

<p>Input: <code>fit</code> · list · Named container; each member has its own dimensions; <code>initial</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

range displays the minimum and maximum across all n fitted g values. Keep the initial plug-in estimate, AIPW and score distinct.

</div>

<div class="r-line-card" data-r-line="72">

#### R line 72

```r
for (epsilon in c(0, 0.002, 0.004)) {
```

<p>Input: constants or the current R environment. Output: <code>epsilon</code> · numeric scalar (double) · Length 1.</p>

Manually try three update coefficients with the full baseline model, then find the root by bisection.

</div>

<div class="r-line-card" data-r-line="74">

#### R line 74

```r
  trial <- full_at(fit, epsilon)
```

<p>Input: <code>fit</code> · list · Named container; each member has its own dimensions; <code>epsilon</code> · numeric scalar (double) · Length 1. Output: <code>trial</code> · list · Named container; each member has its own dimensions.</p>

For each patient, calculate the update using fixed offset plus epsilon×H, then calculate the score and mean log likelihood.

</div>

<div class="r-line-card" data-r-line="76">

#### R line 76

```r
  cat("full trial epsilon =", epsilon, "; score =", trial$score, "; log likelihood =", trial$log_likelihood, "\n")
```

<p>Input: <code>epsilon</code> · numeric scalar (double) · Length 1; <code>trial</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the mean score and mean log likelihood over all 5,735 patients; the denominator is the full cohort size.

</div>

<div class="r-line-card" data-r-line="78">

#### R line 78

```r
}
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

End the manual trials for the full baseline model.

</div>

<div class="r-line-card" data-r-line="80">

#### R line 80

```r
lo <- -0.25
```

<p>Input: constants or the current R environment. Output: <code>lo</code> · numeric scalar (double) · Length 1.</p>

Set the lower bisection bound for the full baseline model’s score.

</div>

<div class="r-line-card" data-r-line="82">

#### R line 82

```r
hi <- 0.25
```

<p>Input: constants or the current R environment. Output: <code>hi</code> · numeric scalar (double) · Length 1.</p>

Set the upper bisection bound for the full baseline model’s score.

</div>

<div class="r-line-card" data-r-line="84">

#### R line 84

```r
stopifnot(full_at(fit, lo)$score >= 0, full_at(fit, hi)$score <= 0)
```

<p>Input: <code>fit</code> · list · Named container; each member has its own dimensions; <code>lo</code> · numeric scalar (double) · Length 1; <code>hi</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Check that the non-increasing score for the full baseline model brackets a root. Stop explicitly if this fails.

</div>

<div class="r-line-card" data-r-line="86">

#### R line 86

```r
full_targeted <- NULL
```

<p>Input: constants or the current R environment. Output: <code>full_targeted</code> · list · Named container; each member has its own dimensions.</p>

Use NULL to mark that no result for the full baseline model has yet met tolerance.

</div>

<div class="r-line-card" data-r-line="88">

#### R line 88

```r
for (iteration in 1:200) {
```

<p>Input: constants or the current R environment. Output: <code>iteration</code> · integer scalar · Length 1.</p>

Start up to 200 explicit bisection steps for the full baseline model.

</div>

<div class="r-line-card" data-r-line="90">

#### R line 90

```r
  step <- full_bisection_step(fit, lo, hi)
```

<p>Input: <code>fit</code> · list · Named container; each member has its own dimensions; <code>lo</code> · numeric scalar (double) · Length 1; <code>hi</code> · numeric scalar (double) · Length 1. Output: <code>step</code> · list · Named container; each member has its own dimensions.</p>

Evaluate the midpoint score and narrow the bracket using the same fit, keeping all initial Q and g values fixed.

</div>

<div class="r-line-card" data-r-line="92">

#### R line 92

```r
  if (iteration <= 5) cat("full bisection", iteration, "; epsilon =", step$midpoint, "; score =", step$score, "\n")
```

<p>Input: <code>iteration</code> · integer scalar · Length 1; <code>step</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the midpoint and score for the first five patient-level bisection steps so readers can check them manually.

</div>

<div class="r-line-card" data-r-line="94">

#### R line 94

```r
  if (abs(step$score) <= tolerance) {
```

<p>Input: <code>step</code> · list · Named container; each member has its own dimensions; <code>tolerance</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Check the full n-patient score against the same prespecified tolerance, 1e−12.

</div>

<div class="r-line-card" data-r-line="96">

#### R line 96

```r
    full_targeted <- full_at(fit, step$midpoint)
```

<p>Input: <code>fit</code> · list · Named container; each member has its own dimensions; <code>step</code> · list · Named container; each member has its own dimensions. Output: <code>full_targeted</code> · list · Named container; each member has its own dimensions.</p>

Name the result targeted only after solving for the root. Both arms use the same epsilon and are standardised over the same n patients.

$$
\widehat\psi_{\rm TMLE}=n^{-1}\sum_i(Q^*_{1,i}-Q^*_{0,i})
$$


</div>

<div class="r-line-card" data-r-line="98">

#### R line 98

```r
    break
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

Exit the bisection loop for the full baseline model.

</div>

<div class="r-line-card" data-r-line="100">

#### R line 100

```r
  }
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

End the convergence branch for the full baseline model.

</div>

<div class="r-line-card" data-r-line="102">

#### R line 102

```r
  stopifnot(step$midpoint != lo, step$midpoint != hi)
```

<p>Input: <code>step</code> · list · Named container; each member has its own dimensions; <code>lo</code> · numeric scalar (double) · Length 1; <code>hi</code> · numeric scalar (double) · Length 1. Output: checks, displayed output or control flow.</p>

Stop immediately when floating-point precision prevents further narrowing, avoiding false convergence.

</div>

<div class="r-line-card" data-r-line="104">

#### R line 104

```r
  lo <- step$lo
```

<p>Input: <code>step</code> · list · Named container; each member has its own dimensions. Output: <code>lo</code> · numeric scalar (double) · Length 1.</p>

Keep the new lower bound of the bracket containing the root.

</div>

<div class="r-line-card" data-r-line="106">

#### R line 106

```r
  hi <- step$hi
```

<p>Input: <code>step</code> · list · Named container; each member has its own dimensions. Output: <code>hi</code> · numeric scalar (double) · Length 1.</p>

Keep the new upper bound of the bracket containing the root.

</div>

<div class="r-line-card" data-r-line="108">

#### R line 108

```r
}
```

<p>Input: constants or the current R environment. Output: checks, displayed output or control flow.</p>

End the full baseline model’s bisection loop.

</div>

<div class="r-line-card" data-r-line="110">

#### R line 110

```r
if (is.null(full_targeted)) stop("Full baseline bisection did not converge")
```

<p>Input: <code>full_targeted</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Report an error when iterations are exhausted; success is defined by the score tolerance.

</div>

<div class="r-line-card" data-r-line="112">

#### R line 112

```r
cat("full epsilon =", full_targeted$epsilon, "; risk0 =", full_targeted$risk0, "; risk1 =", full_targeted$risk1, "; TMLE =", full_targeted$estimate, "\n")
```

<p>Input: <code>full_targeted</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the solved coefficient, each arm’s mean risk over all n patients, and the standardised risk difference.

</div>

<div class="r-line-card" data-r-line="114">

#### R line 114

```r
cat("full score =", full_targeted$score, "; EIF mean =", full_targeted$eif_mean, "; empirical SE =", full_targeted$se, "\n")
```

<p>Input: <code>full_targeted</code> · list · Named container; each member has its own dimensions. Output: checks, displayed output or control flow.</p>

Display the likelihood score, full EIF mean and empirical SE separately. See this chapter’s technical notes for finite-sample and inference conditions.

</div>

</details>

</div>

## Change Q and g to check double robustness

<div class="lab" data-lab="8"></div>

This hypothetical experiment fixes the empirical distribution represented by the real joint frequency table as its reference and changes $Q,g$ under controlled conditions to check the population double-robustness identity. Predict first: if only $g$ is correct, must $Q$ be restored point by point? If both are wrong, does a zero score guarantee the correct target?

<details><summary>Answer and explanation</summary>

When only $g$ is correct, the bias product is zero: the target can be correct while $Q$ remains wrong pointwise. If both are wrong, root-finding only solves the residual equation along the path; the updated bias product may still be non-zero. The experiment compares three independent calculations side by side: plug-in using new $Q$, residual correction and the bias identity.

</details>

<details><summary>How can this calculation exercise become a complete clinical research analysis?</summary>

Refine the clinical design, treatment window and target population; verify pretreatment measurement of acute severity; check non-linearity/interactions, weights and overlap; consider multicentre dependence and sensitivity to unmeasured confounding. Loss to follow-up or time-varying treatment requires an expanded observation model and EIF. More flexible learners and cross-fitting also require their own rate conditions. These steps extend the calculation exercise into a complete clinical research analysis.

</details>

## Review the complete calculation from raw data to AIPW and TMLE

You can now use the same real source file to obtain mean mortality, $p_X/g/Q$, the target gradient, checks in every finite direction, projections and empirical variance. You can then construct the offset and $H$, solve for $\epsilon$ manually, update predictions under both treatments and average their differences. This completes the statistical calculation and mathematical verification. Clinical causal inference additionally requires validation of identification assumptions, learner rates and interval coverage.
