How to Efficiently Skip to a Specific Line in a Large Text File?

Skipping to a Specific Line in a Large Text File

When dealing with massive text files, it's common to encounter the need to jump directly to a particular line. One straightforward approach involves iteratively reading through the file, but this can be inefficient if you know the target line number in advance.

The Problem:

Using the code snippet provided:

startFromLine = 141978
urlsfile = open(filename, "rb", 0)
linesCounter = 1
for line in urlsfile:
    if linesCounter > startFromLine:
        DoSomethingWithThisLine(line)
    linesCounter += 1

This code reads the file line by line, requiring processing of all preceding lines, even though they are irrelevant.

A More Efficient Solution:

To efficiently jump to a specific line, you can utilize a two-step approach:

1. Build an Offset List:

   • Read the file line by line, creating a list line_offset that stores the offset of each line start.

2. Seek to Target Line:

   • Given the target line number n, use file.seek(line_offset[n]) to move the file pointer to the start of that line.

Here is an example:

line_offset = []
offset = 0
for line in file:
    line_offset.append(offset)
    offset += len(line)
file.seek(0)

target_line = 141978
file.seek(line_offset[target_line])

This approach allows you to skip directly to the desired line, avoiding the unnecessary overhead of processing irrelevant lines.

The above is the detailed content of How to Efficiently Skip to a Specific Line in a Large Text File?. For more information, please follow other related articles on the PHP Chinese website!