# Handling Emojis and Unicode in YouTube Data

YouTube content frequently contains emojis, special Unicode characters, and text in various languages. The `tuber` package provides built-in functions for detecting, extracting, and manipulating emojis without external dependencies.

## Quick Start

    library(tuber)

    # Get comments from a video
    comments <- get_all_comments(video_id = "your_video_id")

    # Check which comments contain emojis
    comments$has_emoji <- has_emoji(comments$text_display)

    # Count emojis per comment
    comments$emoji_count <- count_emojis(comments$text_display)

    # Filter to emoji-rich comments
    emoji_comments <- comments[comments$emoji_count > 0, ]

## Emoji Detection Functions

The package provides five main functions for working with emojis:

### `has_emoji()` - Check for emoji presence

    has_emoji("Hello world")
    # FALSE

    has_emoji("Great video! \U0001F44D")
    # TRUE

    has_emoji(c("No emoji", "Has emoji \U0001F600", "Also none"))
    # c(FALSE, TRUE, FALSE)

### `count_emojis()` - Count emojis in text

    count_emojis("Hello world")
    # 0

    count_emojis("Rating: \U0001F600\U0001F600\U0001F600")
    # 3

    count_emojis(c("None", "\U0001F44D", "\U0001F600\U0001F601"))
    # c(0, 1, 2)

### `extract_emojis()` - Get emojis from text

    extract_emojis("Hello \U0001F44B World \U0001F30D!")
    # list(c("\U0001F44B", "\U0001F30D"))

    extract_emojis(c("No emoji", "\U0001F600\U0001F601"))
    # list(character(0), c("\U0001F600", "\U0001F601"))

### `remove_emojis()` - Strip emojis from text

    remove_emojis("Hello \U0001F44B World!")
    # "Hello  World!"

    remove_emojis(c("No emoji", "Has \U0001F600 emoji"))
    # c("No emoji", "Has  emoji")

### `replace_emojis()` - Substitute emojis

    replace_emojis("Hello \U0001F44B World!", replacement = "[emoji]")
    # "Hello [emoji] World!"

    replace_emojis("Rate: \U0001F600\U0001F600\U0001F600", replacement = "*")
    # "Rate: ***"

## Common Use Cases

### Filter comments with high emoji usage

    comments <- get_all_comments(video_id = "your_video_id")
    comments$emoji_count <- count_emojis(comments$text_display)

    # Top 10 most emoji-heavy comments
    top_emoji <- comments[order(-comments$emoji_count), ][1:10, ]

### Text analysis without emojis

    # Remove emojis for text analysis
    comments$clean_text <- remove_emojis(comments$text_display)

    # Now use clean_text for sentiment analysis or word clouds

### Emoji frequency analysis

    # Extract all emojis from comments
    all_emojis <- unlist(extract_emojis(comments$text_display))

    # Count frequency
    emoji_freq <- table(all_emojis)
    sort(emoji_freq, decreasing = TRUE)[1:10]

## Unicode Text Processing

Beyond emojis, `tuber` handles Unicode text consistently:

### `safe_utf8()` - Ensure UTF-8 encoding

    problematic_text <- c("caf\xe9", "na\xefve")
    safe_text <- safe_utf8(problematic_text)

### `clean_youtube_text()` - Clean HTML and normalize text

    raw_text <- "Great video! &lt;3 &amp; more..."
    clean_text <- clean_youtube_text(raw_text)
    # "Great video! <3 & more..."

## Troubleshooting

### Emojis appear as question marks

Your R environment may not support UTF-8 display. The data is still correct; only the display is affected. Try:

    # Check locale
    Sys.getlocale("LC_CTYPE")

    # Set UTF-8 locale on macOS/Linux
    Sys.setlocale("LC_CTYPE", "en_US.UTF-8")

### Emoji counts seem too high

Compound emojis (like family emojis or skin tone modifiers) may be counted as multiple characters. This is due to how Unicode encodes these as sequences of code points.

### Some emojis not detected

The emoji pattern covers most common Unicode emoji blocks. Very new emojis added in recent Unicode versions may not be detected until the pattern is updated.