Unlocking Text Files: Python's Powerhouse for Line-by-Line Processing

Open the file:

• Use the open() function to open the file. You'll provide the file path and mode (usually 'r' for reading).

with open("my_file.txt", "r") as file:
  # Read the file here

The with statement ensures the file gets closed properly even if errors occur.

Read the file line-by-line:

There are two common methods:

• Using readlines():

  • This method reads all the lines of the file at once and returns them as a list of strings.

  with open("my_file.txt", "r") as file:
    lines = file.readlines()
  

  Note: This might not be ideal for very large files as it can consume a lot of memory.

• Using a loop with readline():

  • This method reads the file one line at a time using a loop. It's more memory-efficient for large files.

  with open("my_file.txt", "r") as file:
    lines = []
    for line in file:
      lines.append(line)
  

Process the lines (optional):

• Once you have the lines in a list, you can iterate over them and process them as needed.

  for line in lines:
    # Do something with each line
    print(line.strip())  # Remove leading/trailing whitespace
  

Here are some additional points to consider:

• By default, readline() includes the newline character (\n) at the end of each line. You can use the strip() method to remove it.
• If the file doesn't exist, open() will raise a FileNotFoundError. You can handle this using a try-except block.

Example 1: Reading a file using readlines()

# This code reads the entire file "my_file.txt" into a list

with open("my_file.txt", "r") as file:
  lines = file.readlines()

# Access lines in the list (assuming lines is defined above)
for line in lines:
  print(line.strip())  # Print each line with leading/trailing whitespace removed

# This code reads "my_file.txt" line by line and stores them in a list

with open("my_file.txt", "r") as file:
  lines = []
  for line in file:
    lines.append(line.strip())  # Remove whitespace while adding each line

# Process the lines in the list (assuming lines is defined above)
for line in lines:
  # Do something with each line (e.g., print or further process)
  print(f"Line: {line}")  # Print each line with a label

Using list comprehension:

List comprehension offers a concise way to create a list from an iterable. Here's how to use it for reading a file:

with open("my_file.txt", "r") as file:
  lines = [line.strip() for line in file]  # Strip whitespace while reading

This approach iterates over the file object directly and creates a new list with each line stripped of whitespace.

Using the itertools.islice function (for specific line ranges):

The itertools module provides the islice function that helps iterate over a specific slice of an iterable. You can use it to read only a certain number of lines:

from itertools import islice

with open("my_file.txt", "r") as file:
  lines = list(islice(file, 10))  # Read the first 10 lines

# You can adjust the start and end index in islice for different ranges

Using generators (memory-efficient for large files):

Generators are functions that return an iterator, yielding one element at a time instead of creating the entire list in memory. This is particularly useful for very large files.

def read_lines(filename):
  with open(filename, "r") as file:
    for line in file:
      yield line.strip()  # Yield each line with whitespace stripped

# Usage
for line in read_lines("my_file.txt"):
  # Process each line here
  print(line)