Skip to content
This repository has been archived by the owner on Jun 10, 2024. It is now read-only.

fix bug in module reload #827

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

KaitoHH
Copy link

@KaitoHH KaitoHH commented Aug 17, 2018

This fixed #149.


Problem

I encountered a module-import error.
For example, if I create a project like this:

# Project: f_project
def f():
    print('f called.')

and in another project, I import f_project and call f() like this:

...
# Project: spider
from projects.f_project import f

class...
    def on_start(self):
        f() # this calls f in f_project
        ...
...

It will output the correct result the first time you run the script, but when you modify the script in f_project, for example, change f called to f called again, and run spider again, you will still get the result as it was. However, restarting pyspider can reload the project.

Analysis

After reading code in \pyspider\processor\project_module.py, I think this bug was produced because pyspider use six.exec_(code, mod.__dict__) to dynamically execute the script, and this code import a same module multiple times whenever pressing the run button. Since in Python, a module can only be imported once, any import after the first import will be ignored, so the result remains the same.

Solution

I'm willing to use something like six.moves.reload_module to reload the module, but it has no effect, so I just simply removed all modules matching projects.* (that means all modules in pyspider project) before ProjectLoader.load_module was called. As a result, all modules in this project will be re-import, and that solved the problem.

Looking forward to any better solution on solving this bug if available.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

How to reload Projects rules
1 participant