readme and pkgdown

Author: ASuGuit

README.Rmd

@@ -0,0 +1,176 @@
+---
+title: The `automatedRecLin` Package
+output: github_document
+---
+
+```{r, include = FALSE}
+knitr::opts_chunk$set(
+  collapse = TRUE,
+  comment = "#>",
+  fig.path = "man/figures/README-",
+  out.width = "100%"
+)
+```
+
+## Description
+
+This R package is designed to perform record linkage (also known as entity resolution) in unsupervised or supervised settings. It compares pairs of records from two datasets using selected comparison functions to estimate the probability or density ratio between matched and non-matched records. Based on these estimates, it predicts a set of matches that maximizes entropy.
+
+## Installation
+
+To install the development version from GitHub you can use the `pak` package.
+
+```{r, eval=FALSE}
+# install.packages("pak") # uncomment if needed
+pak::pkg_install("ncn-foreigners/automatedRecLin")
+```
+
+## Basic usage
+
+Load the package for the examples.
+
+```{r}
+library(automatedRecLin)
+```
+
+### Unsupervised maximum entropy classifier for record linkage
+
+Generate two simple datasets that contain some common records, with typos in some cases.
+
+```{r}
+df_1 <- data.frame(
+  name = c("Emma", "Liam", "Olivia", "Noah", "Ava",
+           "Ethan", "Sophia", "Mason", "Isabella", "James"),
+  surname = c("Smith", "Johnson", "Williams", "Brown", "Jones",
+              "Garcia", "Miller", "Davis", "Rodriguez", "Wilson"),
+  city = c("New York", "Los Angeles", "Chicago", "Houston", "Phoenix",
+           "Philadelphia", "San Antonio", "San Diego", "Dallas", "San Jose")
+)
+
+df_2 <- data.frame(
+  name = c(
+    "Emma", "Liam", "Olivia", "Noah",
+    "Ava", "Ehtan", "Sopia", "Mson",
+    "Charlotte", "Benjamin", "Amelia", "Lucas"
+  ),
+  surname = c(
+    "Smith", "Johnson", "Williams", "Brown",
+    "Jnes", "Garca", "Miler", "Dvis",
+    "Martinez", "Lee", "Hernandez", "Clark"
+  ),
+  city = c(
+    "New York", "Los Angeles", "Chicago", "Houston",
+    "Phonix", "Philadelpia", "San Antnio", "San Dieg",
+    "Seattle", "Miami", "Boston", "Denver"
+  )
+)
+df_1
+df_2
+```
+
+Specify key variables used for record linkage. Select a comparison function (i.e. a function to compare pairs of records) for each variable. For example, use the `jarowinkler_complement` function from the `automatedRecLin` package (1 - Jaro-Winkler distance). Choose a method for estimating the probability or density ratio for each variable. The available methods are: `"binary"`, `"continuous_parametric"` and `"continuous_nonparametric"`.
+
+```{r}
+variables <- c("name", "surname", "city")
+comparators <- list(
+  "name" = jarowinkler_complement(),
+  "surname" = jarowinkler_complement(),
+  "city" = jarowinkler_complement()
+)
+methods <- list(
+  "name" = "continuous_parametric",
+  "surname" = "continuous_parametric",
+  "city" = "continuous_parametric"
+)
+```
+
+Perform record linkage using the `mec` function. The output contains the following information:
+
++ the names of key variables,
++ the number of predicted matches,
++ the first 6 predicted matches (with their estimated probability or density ratio),
++ the method for constructing the predicted set of matches (default: `"size"`),
++ estimated false link rate (FLR),
++ estimated missing match rate (MMR),
++ estimated parameters for variables using the `"binary"` or `"continuous_parametric"` methods.
+
+```{r}
+set.seed(1)
+unsup_result <- mec(A = df_1, B = df_2,
+                    variables = variables,
+                    comparators = comparators,
+                    methods = methods)
+unsup_result
+```
+
+### Supervised maximimum entropy classifier for record linkage
+
+Generate two simple training datasets that contain some common records, with typos in some cases.
+
+```{r}
+df_1_train <- data.frame(
+        "name" = c("John", "Emily", "Mark", "Anna", "David"),
+        "surname" = c("Smith", "Johnson", "Taylor", "Williams", "Brown")
+)
+df_2_train <- data.frame(
+        "name" = c("John", "Emely", "Marc", "Michael"),
+        "surname" = c("Smith", "Jonson", "Tailor", "Henderson")
+)
+df_1_train
+df_2_train
+```
+
+Specify the key variables, select comparison functions and choose methods for estimating the probability or density ratio. Additionally, provide a `data.frame` indicating known matches.
+
+```{r}
+variables_train <- c("name", "surname")
+comparators_train <- list("name" = jarowinkler_complement(),
+                          "surname" = jarowinkler_complement())
+methods_train <- list("name" = "continuous_nonparametric",
+                      "surname" = "continuous_nonparametric")
+matches_train <- data.frame("a" = 1:3, "b" = 1:3)
+```
+
+Train a record linkage model using the `train_rec_lin` function. 
+
+```{r}
+model <- train_rec_lin(A = df_1_train, B = df_2_train,
+                       matches = matches_train,
+                       variables = variables_train,
+                       comparators = comparators_train,
+                       methods = methods_train)
+model
+```
+
+Generate two new datasets for record linkage prediction.
+
+```{r}
+df_1_new <- data.frame(
+  "name" = c("Jame", "Lia", "Tomas", "Matthew", "Andrew"),
+  "surname" = c("Wilsen", "Thomsson", "Davis", "Robinson", "Scott")
+)
+df_2_new <- data.frame(
+  "name" = c("James", "Leah", "Thomas", "Mathew", "Andrew", "Sophie"),
+  "surname" = c("Wilson", "Thompson", "Davies", "Robins", "Scots", "Clarks")
+)
+df_1_new
+df_2_new
+```
+
+Predict matches using the `predict` function. The output has a similar structure to that of the `mec` function.
+
+```{r}
+predict(model, df_1_new, df_2_new)
+```
+
+## Funding
+
+Work on this package is supported by the National Science Centre, OPUS 20 grant no. 2020/39/B/HS4/00941 (Towards census-like statistics for foreign-born populations -- quality, data integration and estimation).
+
+## References
+
+Lee, D., Zhang, L.-C. and Kim, J. K. (2022). [Maximum entropy classification for record linkage.](https://www150.statcan.gc.ca/n1/pub/12-001-x/2022001/article/00007-eng.htm) Survey Methodology, Statistics Canada, Catalogue No. 12-001-X, Vol. 48, No. 1. 
+
+Vo, T. H., Chauvet, G., Happe, A., Oger, E., Paquelet, S., and Garès, V. (2023). [Extending the Fellegi-Sunter record linkage model for mixed-type data with application to the French national health data system.](https://ideas.repec.org/a/eee/csdana/v179y2023ics0167947322002365.html) Computational Statistics & Data Analysis, 179, 107656.
+
+Sugiyama, M., Suzuki, T., Nakajima, S. et al. [Direct importance estimation for covariate shift adaptation.](https://doi.org/10.1007/s10463-008-0197-x) Ann Inst Stat Math 60, 699–746 (2008).