python example of using the most efficient way to delete a certain line in a large file

The following editor will bring you an article about deleting a certain line in a large file in python (the most efficient method). The editor thinks it’s pretty good, so I’ll share it with you now and give it as a reference. Let's follow the editor and take a look.

When using python to process a text and want to delete a certain line in it, the conventional idea is to read the file into the memory first, modify it in the memory, and then write it to the source file. .

But if you want to process a large text, such as GB-level text, this method not only requires a large amount of memory, but also takes time to read into the memory at one time, and may also cause memory overflow.

So, we need to use another way of thinking to deal with it.

We can use the open() method to open the file that needs to be modified into two files, and then read it into the memory line by line. When the line that needs to be deleted is found, overwrite it one by one with the following lines. See the code below for implementation.

with open('file.txt', 'r') as old_file:
  with open('file.txt', 'r+') as new_file:

    current_line = 0

    # 定位到需要删除的行
    while current_line < (del_line - 1):
      old_file.readline()
      current_line += 1

    # 当前光标在被删除行的行首，记录该位置
    seek_point = old_file.tell()

    # 设置光标位置
    new_file.seek(seek_point, 0)

    # 读需要删除的行，光标移到下一行行首
    old_file.readline()
    
    # 被删除行的下一行读给 next_line
    next_line = old_file.readline()

    # 连续覆盖剩余行，后面所有行上移一行
    while next_line:
      new_file.write(next_line)
      next_line = old_file.readline()

    # 写完最后一行后截断文件，因为删除操作，文件整体少了一行，原文件最后一行需要去掉
    new_file.truncate()

The above is the detailed content of python example of using the most efficient way to delete a certain line in a large file. For more information, please follow other related articles on the PHP Chinese website!