vegibit

Python csv Tutorial

Click to share! ⬇️

The Python CSV module is a built-in library that provides functionality to read from and write to CSV (Comma Separated Values) files. CSV files are a widely used data storage format, as they are simple, easy to read, and can be opened in various applications, including spreadsheet software like Microsoft Excel or Google Sheets.

CSV files store data in plain text format, with each row representing a record and values separated by a delimiter, typically a comma. However, other delimiters such as tabs, spaces, or semicolons can also be used. The first row often contains the header, which describes the field names for the columns of data.

The Python CSV module offers various classes and functions to work with CSV files, such as:

  1. csv.reader: A class to read data from a CSV file. It takes a file object as an argument and returns an iterator that produces lists for each row in the file.
  2. csv.writer: A class to write data to a CSV file. It takes a file object as an argument and provides methods to write rows or individual values to the file.
  3. csv.DictReader: A class to read data from a CSV file into dictionaries. It maps the data in each row to the corresponding field names specified in the header or provided as a separate argument.
  4. csv.DictWriter: A class to write data to a CSV file from dictionaries. It writes dictionaries as rows in the file, mapping the keys to the appropriate field names in the header.

By using the Python CSV module, you can efficiently manage CSV files in your Python programs, enabling you to import, process, and export data in a structured and standardized manner.

What is CSV File Format

CSV (Comma Separated Values) file format is a simple, widely used data storage format that stores tabular data in plain text. Each line in a CSV file represents a row or a record, and the values in each row are separated by a delimiter, typically a comma. However, other delimiters like tabs, spaces, or semicolons can also be used depending on the data and application requirements.

The simplicity and versatility of the CSV file format make it a popular choice for data exchange between applications, systems, and platforms. It is commonly used for storing and sharing spreadsheet or database data, and can be easily opened, viewed, and edited using spreadsheet software like Microsoft Excel, Google Sheets, or LibreOffice Calc, as well as text editors and programming languages.

A CSV file usually consists of the following components:

  1. Header: The first row of a CSV file often contains the header, which lists the field names or column labels for the data. The header is optional, but it helps provide context and structure to the data.
  2. Rows: Each subsequent row in the file represents a single record or data entry. Rows are separated by newline characters.
  3. Columns: Within each row, the values are separated by a delimiter, creating columns. The number of columns should remain consistent across all rows in a well-structured CSV file.
  4. Delimiter: A character used to separate the values in each row. While commas are the most common delimiter, other characters such as tabs, spaces, or semicolons can also be used depending on the data and requirements.

Here’s an example of a simple CSV file:

Name,Age,Occupation
Alice,30,Engineer
Bob,25,Designer
Charlie,22,Student

In this example, the header row lists the field names “Name,” “Age,” and “Occupation.” The subsequent rows contain the data, with each value separated by a comma.

How to Read CSV Files in Python

Reading CSV files in Python is easy using the built-in csv module. In this section, we’ll go through a step-by-step guide on how to read CSV files using the csv.reader class.

Step 1: Import the CSV module

First, you need to import the csv module:

import csv

Step 2: Open the CSV file

Use the open() function to open the CSV file you want to read. Remember to specify the correct file path and the mode in which you want to open the file. Use the 'r' mode for reading.

file = open('example.csv', 'r')

Step 3: Create a CSV reader

Next, create a CSV reader using the csv.reader() function. Pass the file object as an argument to the function. You can also specify additional options like the delimiter, quote character, and more.

csv_reader = csv.reader(file)

Step 4: Iterate over the rows

Now, you can iterate over the rows in the CSV file using a for loop. The csv.reader object returns an iterator that produces lists for each row in the file.

for row in csv_reader:
    print(row)

Each row will be a list containing the values in the corresponding row of the CSV file.

Step 5: Close the file

After you finish reading the CSV file, it’s a good practice to close the file using the close() method.

file.close()

Complete Example

Here’s the complete example code for reading a CSV file:

import csv

# Open the file
file = open('example.csv', 'r')

# Create a CSV reader
csv_reader = csv.reader(file)

# Iterate over the rows
for row in csv_reader:
    print(row)

# Close the file
file.close()

Alternatively, you can use the with statement to handle file opening and closing more efficiently:

import csv

# Open the file with a context manager
with open('example.csv', 'r') as file:
    # Create a CSV reader
    csv_reader = csv.reader(file)

    # Iterate over the rows
    for row in csv_reader:
        print(row)

With this approach, the file is automatically closed when the with block is exited, making the code more concise and easier to read.

Examples of Reading CSV Files with Different Delimiters

In some cases, CSV files may use different delimiters instead of the common comma (,). For example, tab-separated values (TSV) files use tabs (\t) as delimiters, while some files may use semicolons (;) or other characters. The csv.reader function allows you to specify the delimiter used in the file with the delimiter parameter. Here are some examples of reading CSV files with different delimiters:

Example 1: Tab-separated values (TSV)

Suppose you have a TSV file named example.tsv with the following content:

Name\tAge\tOccupation
Alice\t30\tEngineer
Bob\t25\tDesigner
Charlie\t22\tStudent

You can read this file using the csv.reader function by setting the delimiter parameter to '\t':

import csv

with open('example.tsv', 'r') as file:
    tsv_reader = csv.reader(file, delimiter='\t')
    for row in tsv_reader:
        print(row)

Example 2: Semicolon-separated values

Consider a CSV file named example_semicolon.csv with the following content:

Name;Age;Occupation
Alice;30;Engineer
Bob;25;Designer
Charlie;22;Student

To read this file, set the delimiter parameter to ';':

import csv

with open('example_semicolon.csv', 'r') as file:
    semicolon_reader = csv.reader(file, delimiter=';')
    for row in semicolon_reader:
        print(row)

Example 3: Custom delimiter

If you have a CSV file that uses a custom delimiter, such as the pipe character (|), you can still read it using the csv.reader function. Suppose you have a file named example_pipe.csv with the following content:

Name|Age|Occupation
Alice|30|Engineer
Bob|25|Designer
Charlie|22|Student

You can read this file by setting the delimiter parameter to the pipe character ('|'):

import csv

with open('example_pipe.csv', 'r') as file:
    pipe_reader = csv.reader(file, delimiter='|')
    for row in pipe_reader:
        print(row)

By specifying the appropriate delimiter, you can easily read CSV files with various delimiters using the csv.reader function in Python.

How to Write CSV Files in Python

Writing CSV files in Python is straightforward using the built-in csv module. In this section, we’ll go through a step-by-step guide on how to write CSV files using the csv.writer class.

Step 1: Import the CSV module

First, you need to import the csv module:

import csv

Step 2: Open the CSV file

Use the open() function to open the CSV file you want to write. Remember to specify the correct file path and the mode in which you want to open the file. Use the 'w' mode for writing. If the file does not exist, it will be created.

file = open('output.csv', 'w')

Step 3: Create a CSV writer

Next, create a CSV writer using the csv.writer() function. Pass the file object as an argument to the function. You can also specify additional options like the delimiter, quote character, and more.

csv_writer = csv.writer(file)

Step 4: Write rows to the CSV file

You can write rows to the CSV file using the writerow() method of the csv.writer object. Pass a list of values as an argument to the method. The values will be written to the file as a single row, separated by the specified delimiter.

csv_writer.writerow(['Name', 'Age', 'Occupation'])
csv_writer.writerow(['Alice', 30, 'Engineer'])
csv_writer.writerow(['Bob', 25, 'Designer'])
csv_writer.writerow(['Charlie', 22, 'Student'])

Alternatively, you can use the writerows() method to write multiple rows at once. Pass a list of lists as an argument to the method, with each inner list representing a row.

rows = [
    ['Name', 'Age', 'Occupation'],
    ['Alice', 30, 'Engineer'],
    ['Bob', 25, 'Designer'],
    ['Charlie', 22, 'Student']
]

csv_writer.writerows(rows)

Step 5: Close the file

After you finish writing to the CSV file, it’s a good practice to close the file using the close() method.

file.close()

Complete Example

Here’s the complete example code for writing a CSV file:

import csv

# Open the file
file = open('output.csv', 'w')

# Create a CSV writer
csv_writer = csv.writer(file)

# Write rows to the file
rows = [
    ['Name', 'Age', 'Occupation'],
    ['Alice', 30, 'Engineer'],
    ['Bob', 25, 'Designer'],
    ['Charlie', 22, 'Student']
]

csv_writer.writerows(rows)

# Close the file
file.close()

You can also use the with statement to handle file opening and closing more efficiently:

import csv

# Open the file with a context manager
with open('output.csv', 'w') as file:
    # Create a CSV writer
    csv_writer = csv.writer(file)

    # Write rows to the file
    rows = [
        ['Name', 'Age', 'Occupation'],
        ['Alice', 30, 'Engineer'],
        ['Bob', 25, 'Designer'],
        ['Charlie', 22, 'Student']
    ]

    csv_writer.writerows(rows)

With this approach, the file is automatically closed when the with block is exited, making the code more concise and easier to read.

What is DictReader and DictWriter in Python CSV

In the Python CSV module, DictReader and DictWriter are two specialized classes for reading and writing CSV files using dictionaries. They provide a more convenient way to work with CSV files when dealing with structured data, as they map the data in each row to the corresponding field names specified in the header or provided as a separate argument.

DictReader

csv.DictReader is a class for reading CSV files into dictionaries. Each row in the CSV file is read as an ordered dictionary, with keys corresponding to the field names (column headers) and values representing the data in each cell. This makes it easier to access and manipulate the data using the field names instead of relying on column indices.

Here’s an example of using csv.DictReader to read a CSV file:

import csv

with open('example.csv', 'r') as file:
    csv_dict_reader = csv.DictReader(file)

    for row in csv_dict_reader:
        print(row)

In this example, each row is an ordered dictionary with keys matching the field names in the CSV file header.

DictWriter

csv.DictWriter is a class for writing CSV files from dictionaries. It writes dictionaries as rows in the file, mapping the keys to the appropriate field names in the header. This is useful when working with structured data, as it allows you to easily write rows with varying data without worrying about the column order.

To use csv.DictWriter, you need to provide the fieldnames (column headers) as a list when creating the writer object. Then, you can use the writeheader() method to write the header row and the writerow() or writerows() methods to write individual rows or multiple rows at once.

Here’s an example of using csv.DictWriter to write a CSV file:

import csv

# Define the field names and data
fieldnames = ['Name', 'Age', 'Occupation']
rows = [
    {'Name': 'Alice', 'Age': 30, 'Occupation': 'Engineer'},
    {'Name': 'Bob', 'Age': 25, 'Occupation': 'Designer'},
    {'Name': 'Charlie', 'Age': 22, 'Occupation': 'Student'}
]

with open('output.csv', 'w') as file:
    csv_dict_writer = csv.DictWriter(file, fieldnames=fieldnames)

    # Write the header
    csv_dict_writer.writeheader()

    # Write the rows
    csv_dict_writer.writerows(rows)

In this example, the csv.DictWriter object writes the header row based on the provided fieldnames and then writes each dictionary in the rows list as a row in the CSV file.

Using DictReader and DictWriter can make your code more readable and easier to maintain, especially when working with structured data that has meaningful field names.

How to Use DictReader to Read CSV Files

Using csv.DictReader to read CSV files makes it easier to work with structured data, as it reads each row into an ordered dictionary with keys corresponding to the field names (column headers). Here’s a step-by-step guide on how to use csv.DictReader:

Step 1: Import the CSV module

First, you need to import the csv module:

import csv

Step 2: Open the CSV file

Use the open() function to open the CSV file you want to read. Remember to specify the correct file path and the mode in which you want to open the file. Use the 'r' mode for reading.

file = open('example.csv', 'r')

Step 3: Create a DictReader object

Create a csv.DictReader object by passing the file object as an argument to the csv.DictReader() function. You can also specify additional options like the delimiter, fieldnames, and more.

csv_dict_reader = csv.DictReader(file)

Step 4: Iterate over the rows

Now, you can iterate over the rows in the CSV file using a for loop. The csv.DictReader object returns an iterator that produces ordered dictionaries for each row in the file.

for row in csv_dict_reader:
    print(row)

Each row is an ordered dictionary with keys matching the field names in the CSV file header and values representing the data in each cell.

Step 5: Close the file

After you finish reading the CSV file, it’s a good practice to close the file using the close() method.

file.close()

Complete Example

Here’s the complete example code for reading a CSV file using csv.DictReader:

import csv

# Open the file
file = open('example.csv', 'r')

# Create a DictReader object
csv_dict_reader = csv.DictReader(file)

# Iterate over the rows
for row in csv_dict_reader:
    print(row)

# Close the file
file.close()

Alternatively, you can use the with statement to handle file opening and closing more efficiently:

import csv

# Open the file with a context manager
with open('example.csv', 'r') as file:
    # Create a DictReader object
    csv_dict_reader = csv.DictReader(file)

    # Iterate over the rows
    for row in csv_dict_reader:
        print(row)

With this approach, the file is automatically closed when the with block is exited, making the code more concise and easier to read.

How to Use DictWriter to Write CSV Files

csv.DictWriter allows you to write CSV files from dictionaries, making it easier to work with structured data. Here’s a step-by-step guide on how to use csv.DictWriter to write CSV files:

Step 1: Import the CSV module

First, you need to import the csv module:

import csv

Step 2: Define the field names and data

Define the field names (column headers) as a list and the data as a list of dictionaries, where each dictionary represents a row with keys corresponding to the field names.

fieldnames = ['Name', 'Age', 'Occupation']
rows = [
    {'Name': 'Alice', 'Age': 30, 'Occupation': 'Engineer'},
    {'Name': 'Bob', 'Age': 25, 'Occupation': 'Designer'},
    {'Name': 'Charlie', 'Age': 22, 'Occupation': 'Student'}
]

Step 3: Open the CSV file

Use the open() function to open the CSV file you want to write. Remember to specify the correct file path and the mode in which you want to open the file. Use the 'w' mode for writing. If the file does not exist, it will be created.

file = open('output.csv', 'w')

Step 4: Create a DictWriter object

Create a csv.DictWriter object by passing the file object and the list of fieldnames as arguments to the csv.DictWriter() function. You can also specify additional options like the delimiter, quote character, and more.

csv_dict_writer = csv.DictWriter(file, fieldnames=fieldnames)

Step 5: Write the header

Use the writeheader() method of the csv.DictWriter object to write the header row based on the provided fieldnames.

csv_dict_writer.writeheader()

Step 6: Write the rows

Use the writerows() method of the csv.DictWriter object to write multiple rows at once. Pass the list of dictionaries containing the data as an argument to the method.

csv_dict_writer.writerows(rows)

Alternatively, you can use the writerow() method to write individual rows one at a time.

for row in rows:
    csv_dict_writer.writerow(row)

Step 7: Close the file

After you finish writing to the CSV file, it’s a good practice to close the file using the close() method.

file.close()

Complete Example

Here’s the complete example code for writing a CSV file using csv.DictWriter:

import csv

fieldnames = ['Name', 'Age', 'Occupation']
rows = [
    {'Name': 'Alice', 'Age': 30, 'Occupation': 'Engineer'},
    {'Name': 'Bob', 'Age': 25, 'Occupation': 'Designer'},
    {'Name': 'Charlie', 'Age': 22, 'Occupation': 'Student'}
]

# Open the file
file = open('output.csv', 'w')

# Create a DictWriter object
csv_dict_writer = csv.DictWriter(file, fieldnames=fieldnames)

# Write the header
csv_dict_writer.writeheader()

# Write the rows
csv_dict_writer.writerows(rows)

# Close the file
file.close()

How to Handle CSV Files with Unicode Characters

When working with CSV files containing Unicode characters, you need to handle encoding and decoding properly to ensure that the text is read and written correctly. The most common Unicode encoding is UTF-8. Here’s a step-by-step guide on how to handle CSV files with Unicode characters using the csv module in Python:

Reading CSV Files with Unicode Characters

  1. Import the necessary modules:
import csv
import io
  1. Open the CSV file with the appropriate encoding (e.g., 'utf-8'):
with io.open('example_unicode.csv', 'r', encoding='utf-8') as file:
  1. Create a csv.reader or csv.DictReader object:
    csv_reader = csv.reader(file)
    # or
    csv_dict_reader = csv.DictReader(file)
  1. Iterate over the rows and process the data:
    for row in csv_reader:
        print(row)
    # or
    for row in csv_dict_reader:
        print(row)

Writing CSV Files with Unicode Characters

  1. Import the necessary modules:
import csv
import io
  1. Open the CSV file with the appropriate encoding (e.g., 'utf-8'):
with io.open('output_unicode.csv', 'w', encoding='utf-8', newline='') as file:
  1. Create a csv.writer or csv.DictWriter object:
    csv_writer = csv.writer(file)
    # or
    fieldnames = ['Name', 'Age', 'Occupation']
    csv_dict_writer = csv.DictWriter(file, fieldnames=fieldnames)
  1. Write the header (if using csv.DictWriter):
    csv_dict_writer.writeheader()
  1. Write the rows with Unicode characters:
    rows = [
        ['José', 30, 'Engineer'],
        ['Émily', 25, 'Designer'],
        ['Søren', 22, 'Student']
    ]

    csv_writer.writerows(rows)
    # or
    dict_rows = [
        {'Name': 'José', 'Age': 30, 'Occupation': 'Engineer'},
        {'Name': 'Émily', 'Age': 25, 'Occupation': 'Designer'},
        {'Name': 'Søren', 'Age': 22, 'Occupation': 'Student'}
    ]

    csv_dict_writer.writerows(dict_rows)

By using the io.open() function with the appropriate encoding, you can ensure that the Unicode characters are read and written correctly. The newline='' argument is used when opening the file for writing to ensure that line endings are handled correctly across different platforms.

Examples of Working with CSV Files in Different Encodings

Working with CSV files in different encodings requires specifying the correct encoding when opening the file. Below are examples of reading and writing CSV files in different encodings using the csv module in Python:

Reading a CSV File in ISO-8859-1 Encoding

import csv
import io

with io.open('example_iso-8859-1.csv', 'r', encoding='ISO-8859-1') as file:
    csv_reader = csv.reader(file)
    
    for row in csv_reader:
        print(row)

Writing a CSV File in ISO-8859-1 Encoding

import csv
import io

data = [
    ['Jörg', 35, 'Scientist'],
    ['François', 28, 'Teacher'],
    ['María', 42, 'Doctor']
]

with io.open('output_iso-8859-1.csv', 'w', encoding='ISO-8859-1', newline='') as file:
    csv_writer = csv.writer(file)
    
    csv_writer.writerows(data)

Reading a CSV File in UTF-16 Encoding

import csv
import io

with io.open('example_utf-16.csv', 'r', encoding='UTF-16') as file:
    csv_reader = csv.reader(file)
    
    for row in csv_reader:
        print(row)

Writing a CSV File in UTF-16 Encoding

import csv
import io

data = [
    ['Дмитрий', 33, 'Developer'],
    ['محمد', 29, 'Architect'],
    ['中島', 38, 'Designer']
]

with io.open('output_utf-16.csv', 'w', encoding='UTF-16', newline='') as file:
    csv_writer = csv.writer(file)
    
    csv_writer.writerows(data)

Remember to replace 'ISO-8859-1' and 'UTF-16' with the desired encoding when working with other encodings. Make sure to use the io.open() function instead of the built-in open() to better handle different encodings and specify the newline='' argument when opening the file for writing to ensure that line endings are handled correctly across platforms.

Click to share! ⬇️

Automate Parsing and Renaming of Multiple Files Using Python How Does Python’s JSON Module Make Working With JSON Data Easier Python Webbrowser Module python listsPython Lists python filesHow To Read And Write Files In Python Commonly Used Python Built In FunctionsCommonly Used Python Built In Functions