CSV Processing Skill

Work with CSV data efficiently.

1. Parse CSV

Python:

import csv
import pandas as pd

# Using csv module
with open('data.csv', 'r') as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row['name'], row['email'])

# Using pandas
df = pd.read_csv('data.csv')
print(df.head())
print(df.describe())

2. Data Cleaning

import pandas as pd

df = pd.read_csv('data.csv')

# Remove duplicates
df = df.drop_duplicates()

# Handle missing values
df = df.fillna(0)
df = df.dropna()

# Fix data types
df['age'] = pd.to_numeric(df['age'], errors='coerce')
df['date'] = pd.to_datetime(df['date'])

# Trim whitespace
df['name'] = df['name'].str.strip()

3. Transform CSV

# Filter rows
df_filtered = df[df['age'] > 18]

# Select columns
df_subset = df[['name', 'email']]

# Add calculated column
df['full_name'] = df['first_name'] + ' ' + df['last_name']

# Group and aggregate
df_grouped = df.groupby('category').agg({
    'sales': 'sum',
    'price': 'mean'
})

# Sort
df_sorted = df.sort_values('age', ascending=False)

4. Merge CSV Files

# Merge CSV files
cat file1.csv > merged.csv
tail -n +2 file2.csv >> merged.csv  # Skip header
tail -n +2 file3.csv >> merged.csv

# Merge with pandas
df1 = pd.read_csv('file1.csv')
df2 = pd.read_csv('file2.csv')

# Concatenate
combined = pd.concat([df1, df2], ignore_index=True)

# Merge (join)
merged = pd.merge(df1, df2, on='id', how='inner')

5. Export CSV

# To CSV
df.to_csv('output.csv', index=False)

# To JSON
df.to_json('output.json', orient='records')

# To Excel
df.to_excel('output.xlsx', index=False)

When to Use This Skill

Use /csv for CSV parsing, data cleaning, transformation, and analysis.