```
library(tidyverse)
library(tidymodels)
library(knitr)
```

# AE 10: Flight delays

## Packages

## Data

For this application exercise we will work with a dataset of 25,000 randomly sampled flights that departed one of three NYC airports (JFK, LGA, EWR) in 2013.

`<- read_csv("data/flight-data.csv") flight_data `

```
Rows: 25000 Columns: 10
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (4): origin, dest, carrier, arr_delay
dbl (4): dep_time, flight, air_time, distance
dttm (1): time_hour
date (1): date
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
```

- Convert
`arr_delay`

to factor with levels`"late"`

(first level) and`"on_time"`

(second level). This variable is our outcome and it indicates whether the flight’s arrival was more than 30 minutes.

- Let’s get started with some data prep: Convert all variables that are character strings to factors.

```
#flight_data <- flight_data %>%
# mutate(
# origin = as.factor(origin),
# carrier = as.factor(carrier),
# dest = as.factor(dest)
# )
<- flight_data %>%
flight_data #go across all columns and convert that are characters to factors
#go across all columns and convert if is.character = TRUE to factors
#go across all columns and if is.character apply as.factor
mutate(across(where(is.character), as.factor))
```

## Modeling prep

- Split the data into testing (75%) and training (25%), and save each subset. (Don’t forget to set seed)

- Specify a logistic regression model that uses the
`"glm"`

engine.

Next, we’ll create two recipes and workflows and compare them to each other.

## Model 1: Everything and the kitchen sink

- Define a recipe that predicts
`arr_delay`

using all variables except for`flight`

and`time_hour`

, which, in combination, can be used to identify a flight. Also make sure this recipe handles dummy coding as well as issues that can arise due to having categorical variables with some levels apparent in the training set but not in the testing set. Call this recipe`flights_rec1`

.

- Create a workflow that uses
`flights_rec1`

and the model you specified.

- Fit the this model to the training data using your workflow and display a tidy summary of the model fit.

- Predict
`arr_delay`

for the testing data using this model.

- Plot the ROC curve and find the area under the curve. Comment on how well you think this model has done for predicting arrival delay.

## Model 2: Let’s be a bit more thoughtful

- Define a new recipe,
`flights_rec2`

, that, in addition to what was done in`flights_rec1`

, adds features for day of week and month based on`date`

and also adds indicators for all US holidays (also based on`date`

). A list of these holidays can be found in`timeDate::listHolidays("US")`

. Once these features are added,`date`

should be removed from the data. Then, create a new workflow, fit the same model (logistic regression) to the training data, and do predictions on the testing data. Finally, draw another ROC curve and find the area under the curve. Compare the predictive performance of this new model to the previous one. Based on the area under the curve statistic, which model does better?

## Putting it altogether

- Create an ROC curve that plots both models, in different colors, and adds a legend indicating which model is which.

Compare two models based on ROC curve. Also, calculate their AUCs, AIC, BIC and construct Drop-in-deviance test to make comparison

Choose the best model and construct a hypothesis testing based on the question you are interested in. Also construct a confidence interval of a coefficient you are interested in. Interpret your result

Check the conditions of your model and make comments.

## Acknowledgement

This exercise was inspired by https://www.tidymodels.org/start/recipes/.