R

Data Manipulation with dplyr in R

📘 Introduction

dplyr is one of the most popular R packages for data manipulation. It provides a consistent, readable, and efficient set of functions for selecting, filtering, arranging, transforming, summarizing, and combining data. Compared to Base R, dplyr offers a cleaner syntax that makes data manipulation easier to write and understand.

Information

dplyr is part of the tidyverse collection of packages and is designed to work seamlessly with data frames and tibbles.

🎯 Why Use dplyr?

Write clean and readable code.
Manipulate large datasets efficiently.
Reduce complex Base R syntax.
Perform data transformations quickly.
Support reproducible data analysis workflows.

📦 Installing and Loading dplyr

Install the package once and load it whenever you need to use its functions.

Install and Load dplyr

install.packages("dplyr")

library(dplyr)

📚 Common dplyr Functions

Function	Purpose
select()	Select columns.
filter()	Filter rows.
arrange()	Sort rows.
mutate()	Create or modify columns.
summarise()	Generate summary statistics.
group_by()	Group data for analysis.
rename()	Rename columns.
distinct()	Remove duplicate rows.

📝 Creating a Sample Dataset

Sample Data Frame

library(dplyr)

students <- data.frame(
  ID = c(101,102,103,104),
  Name = c("Alice","Bob","Charlie","David"),
  Marks = c(85,72,91,78),
  Department = c("IT","CS","IT","AI")
)

print(students)

📋 Selecting Columns

The select() function extracts one or more columns from a dataset.

Using select()

students %>%
  select(Name, Marks)

🔍 Filtering Rows

The filter() function returns rows that satisfy specified conditions.

Using filter()

students %>%
  filter(Marks >= 80)

📊 Sorting Data

Use the arrange() function to sort rows in ascending or descending order.

Ascending

Descending

Ascending Order

students %>%
  arrange(Marks)

Descending Order

students %>%
  arrange(desc(Marks))

➕ Creating New Columns

The mutate() function adds new columns or modifies existing ones.

Using mutate()

students %>%
  mutate(
    Grade = ifelse(
      Marks >= 80,
      "A",
      "B"
    )
  )

📈 Summarizing Data

The summarise() function computes summary statistics for a dataset.

Using summarise()

students %>%
  summarise(
    AverageMarks = mean(Marks),
    HighestMarks = max(Marks),
    LowestMarks = min(Marks)
  )

👥 Grouping Data

Combine group_by() and summarise() to calculate summaries for each group.

Group-wise Summary

students %>%
  group_by(Department) %>%
  summarise(
    AverageMarks = mean(Marks)
  )

✏️ Renaming Columns

Using rename()

students %>%
  rename(
    StudentName = Name,
    StudentMarks = Marks
  )

🧹 Removing Duplicate Rows

The distinct() function removes duplicate observations.

Using distinct()

students %>%
  distinct()

📌 Selecting Specific Rows

The slice() function selects rows based on their position.

Using slice()

students %>%
  slice(1:2)

🔢 Counting Observations

The count() function counts observations within groups.

Using count()

students %>%
  count(Department)

🔗 Using the Pipe Operator

The pipe operator %>% passes the output of one operation as the input to the next, making code easier to read.

Pipe Operator Example

students %>%
  filter(Marks >= 80) %>%
  arrange(desc(Marks)) %>%
  select(Name, Marks)

🌍 Real-World Example

A company wants to find the average salary for each department and display the departments in descending order of average salary.

Department Salary Analysis

employees <- data.frame(
  Name = c(
    "Alice",
    "Bob",
    "Charlie",
    "David"
  ),
  Department = c(
    "IT",
    "IT",
    "HR",
    "HR"
  ),
  Salary = c(
    65000,
    70000,
    55000,
    60000
  )
)

employees %>%
  group_by(Department) %>%
  summarise(
    AverageSalary = mean(Salary)
  ) %>%
  arrange(desc(AverageSalary))

🔄 dplyr Workflow

Import Data

Filter Data

Transform Data

Group Records

Summarize Results

Display Output

📋 Base R vs dplyr

Feature	Base R	dplyr
Syntax	More verbose.	Simple and readable.
Performance	Good for small datasets.	Optimized for larger datasets.
Readability	Can become complex.	Highly readable using pipes.
Learning Curve	Moderate.	Easy once the core verbs are understood.

⚠️ Common Mistakes

Mistake	Explanation	Solution
Forgetting to load dplyr	Functions such as filter() and select() will not be available.	Run library(dplyr) before using the package.
Using summarise() without group_by()	Returns a summary for the entire dataset.	Use group_by() when group-wise summaries are needed.
Ignoring the pipe operator	Results in longer and less readable code.	Use %>% to chain multiple operations.
Misspelling column names	Produces errors during execution.	Verify column names using names() or glimpse().

💡 Best Practices

Load dplyr before using its functions.
Use the pipe operator to improve readability.
Combine group_by() and summarise() for grouped analysis.
Use descriptive variable and column names.
Inspect datasets before and after transformations.

Best Practice

The dplyr package provides a consistent set of data manipulation verbs that make data cleaning and transformation faster, more readable, and easier to maintain. It is one of the most widely used tools in modern R data analysis.

📝 Summary

dplyr is a powerful package for data manipulation in R that simplifies common tasks such as selecting columns, filtering rows, sorting data, creating new variables, grouping records, summarizing information, renaming columns, and removing duplicates. By combining these functions with the pipe operator, you can build clear, efficient, and reproducible data analysis workflows suitable for both small and large datasets.

>>"dplyr transforms complex data manipulation into simple, readable, and efficient workflows."