Different Ways to Fix Circular Imports in Python

Have you ever come across circular imports in Python? Well, it’s a very common code smell that indicates something’s wrong with the design or structure.

Circular Import Example

How does circular import occur? This import error usually occurs when two or more modules depending on each other try to import before fully initializing.

Let’s say we have two modules: module_1.py and module_2.py.

# module_1.py
from module_2 import ModY
class ModX:
    mody_obj = ModY()

# module_2.py
from module_1 import ModX
class ModY:
    modx_obj = ModX()

In the above code snippets, both module_1 and module_2 are mutually dependent on each other.

The initialization of mody_obj in module_1 depends on module_2 and the initialization of modx_obj in module_2 depends on module_1.

This is what we call a circular dependency. Both modules will stuck in the import loops while attempting to load each other.

If we run module_1.py, we’ll get the following traceback.

Traceback (most recent call last):
  File "module_1.py", line 1, in <module>
    from module_2 import ModY
  File "module_2.py", line 1, in <module>
    from module_1 import ModX
  File "module_1.py", line 1, in <module>
    from module_2 import ModY
ImportError: cannot import name 'ModY' from partially initialized module 'module_2' (most likely due to a circular import)

This error explains the situation of circular import. When the program attempted to import ModY from module_2, at that time module_2 wasn’t fully initialized (due to another import statement that attempts to import ModX from module_1).

How to fix circular imports in Python? There are different ways to get rid of circular imports in Python.

Fix Circular Imports in Python

Move code into a common file

We can move the code into a common file to avoid import errors and then try to import the modules from that file.

# main.py ----> common file
class ModX:
    pass

class ModY:
    pass

In the above code snippet, we moved the classes ModX and ModY into a common file (main.py).

# module_1.py
from main import ModY

class Mod_X:
    mody_obj = ModY()

# module_2.py
from main import ModX

class Mod_Y:
    modx_obj = ModX()

Now, module_1 and module_2 import the classes from main which fixes the circular import situation.

There is a problem with this approach, sometimes the codebase is so large that it becomes risky to move the code into another file.

Move the import to the end of the module

We can shift the import statement at the end of the module. This will give time to fully initialize the module before importing another module.

# module_1.py
class ModX:
   pass

from module_2 import ModY

class Mod_X:
   mody_obj = ModY()

# module_2.py
class ModY:
   pass

from module_1 import ModX

Importing module within the class/function scope

Importing modules within the class or function scope can avoid circular imports. This allows the module to be imported only when the class or function is invoked. It’s relevant when we want to minimize memory use.

# module_1.py
class ModX:
  pass

class Mod_X:
   from module_2 import ModY
   mody_obj = ModY()

# module_2.py
class ModY:
   pass

class Mod_Y:
   from module_1 import ModX
   modx_obj = ModX()

We moved the import statements within classes Mod_X and Mod_Y scope in module_1 and module_2 respectively.

If we run either module_1 or module_2, we’ll not get a circular import error. But, this approach makes the class accessible only within the class’s scope, so we can’t leverage the import globally.

Using module name/alias

Using the module name or just an alias like this solves the problem. This allows both modules to load fully by deferring circular dependency until runtime.

# module_1.py
from module_2 import ModY
class ModX:
    mody_obj = ModY()

# module_2.py
from module_1 import ModX
class ModY:
    modx_obj = ModX()

Using importlib library

We can also use the importlib library to import the modules dynamically.

Traceback (most recent call last):
  File "module_1.py", line 1, in <module>
    from module_2 import ModY
  File "module_2.py", line 1, in <module>
    from module_1 import ModX
  File "module_1.py", line 1, in <module>
    from module_2 import ModY
ImportError: cannot import name 'ModY' from partially initialized module 'module_2' (most likely due to a circular import)

# main.py ----> common file
class ModX:
    pass

class ModY:
    pass

Circular Imports in Python Packages

Usually, circular imports come from modules within the same package. In complex projects, the directory structure is also complex, with packages within packages.

These packages and sub-packages contain __init__.py files to provide easier access to modules. That’s where sometimes arises circular dependencies among modules unintentionally.

We have the following directory structure.

# module_1.py
from main import ModY

class Mod_X:
    mody_obj = ModY()

We have a package mainpkg and a main.py file. We have two sub-packages modpkg_x and modpkg_y within mainpkg.

Here’s what each Python file within modpkg_x and modpkg_y looks like.

mainpkg/modpkg_x/__init__.py

# module_2.py
from main import ModX

class Mod_Y:
    modx_obj = ModX()

This file imports both classes (ModX and ModA) from module_1 and module_1_1.

mainpkg/modpkg_x/module_1.py

# module_1.py
class ModX:
   pass

from module_2 import ModY

class Mod_X:
   mody_obj = ModY()

The module_1 imports a class ModY from module_2.

mainpkg/modpkg_x/module_1_1.py

# module_2.py
class ModY:
   pass

from module_1 import ModX

The module_1_1 imports nothing. It is not dependent on any module.

mainpkg/modpkg_y/__init__.py

# module_1.py
class ModX:
  pass

class Mod_X:
   from module_2 import ModY
   mody_obj = ModY()

This file imports the class ModY from module_2.

mainpkg/modpkg_y/module_2.py

# module_2.py
class ModY:
   pass

class Mod_Y:
   from module_1 import ModX
   modx_obj = ModX()

The module_2 imports a class ModA from the module_1_1.

We have the following code within the main.py file.

root_dir/main.py

# module_1.py
import module_2 as m2

class ModX:
    def __init__(self):
        self.mody_obj = m2.ModY()

The main file imports a class ModY from module_2. This file is dependent on module_2.

If we visualize the import cycle here, it would look like the following ignoring the __init__.py files within the modpkg_x and modpkg_y.

We can see that the main file depends on module_2, module_1 also depends on module_2 and module_2 depends on module_1_1. There is no import cycle.

But you know, modules depend on their __init__.py file, so the __init__.py file initializes first, and modules are re-imported.

This is what the import cycle looks like now.

This made module_1_1 depend on module_1, which is a fake dependency.

If this is the case, empty the sub-packages __init__.py files and using a separate __init__.py file can help by centralizing imports at the package level.

# module_1.py
from module_2 import ModY
class ModX:
    mody_obj = ModY()

In this structure, we added another sub-package subpkg within mainpkg.

mainpkg/subpkg/__init__.py

# module_2.py
from module_1 import ModX
class ModY:
    modx_obj = ModX()

This will allow internal modules to import from a single source, reducing the need for cross-imports.

Now we can update the import statement within the main.py file.

root_dir/main.py

Traceback (most recent call last):
  File "module_1.py", line 1, in <module>
    from module_2 import ModY
  File "module_2.py", line 1, in <module>
    from module_1 import ModX
  File "module_1.py", line 1, in <module>
    from module_2 import ModY
ImportError: cannot import name 'ModY' from partially initialized module 'module_2' (most likely due to a circular import)

This solves the problem of circular dependency between the modules within the same package.

Conclusion

Circular dependency or import in Python is a code smell which is an indication of serious re-structuring and refactoring of the code.

You can try any of these above-mentioned ways to avoid circular dependency in Python.

?Other articles you might be interested in if you liked this one

✅Template Inheritance in Flask with Example.

✅Difference between exec() and eval() with Examples.

✅Understanding the Use of global Keyword in Python.

✅Python Type Hints: Functions, Return Values, Variable.

✅Why Slash and Asterisk Used in Function Definition.

✅How does the learning rate affect the ML and DL models?

That’s all for now.

Keep Coding✌✌.

The above is the detailed content of Different Ways to Fix Circular Imports in Python. For more information, please follow other related articles on the PHP Chinese website!