background

In an ancient system, there is such a code:

scope = dict(globals(), **locals())

exec(
"""
global_a = 123
def func_a():
    print(global_a)
"""
, scope)
exec("func_a()", scope)

The first user code defines the function, and the second user code executes the function (don't ask why, because the user is always right). After the first code segment is executed, func_a and global_a will be added to the scope. Since the second code segment also uses the same scope, the second code segment calls func_a can output 123 correctly.

However, using exec to execute user code is not elegant and dangerous after all, so the exec function is encapsulated in a python sandbox environment (simply understood as another Python service. After passing code and scope to the service, the service will call exec(code,scope) to execute code in the sandbox environment), which is equivalent to replacing each call to exec with an RPC request to the sandbox service.

So the code looks like this:

scope = dict(globals(), **locals())

scope = call_sandbox(
"""
global_a = 123
def func_a():
    print(global_a)
"""
, scope)
call_sandbox("func_a()", scope)

Scope cross service delivery problem

Because multiple RPC calls need to use the same scope, the sandbox service returns a new scope to ensure that the scope will not be lost in the next call. But executing the code will find the second call_ When sandbox is called, an error will be returned:

global name 'global_a' is not defined

Firstly, it is suspected that the scope is not updated after the first call, but if the scope is not updated, it should report that func cannot be found_ A is right. This error indicates that func in the scope is called the second time_ A exists, but func_a cannot find the variable global_a. By outputting the second call_ Global will be found in the scope before sandbox_ A and func_a all exist:

print(scope.keys())
# ['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__file__', '__cached__', 
# '__builtins__', 'global_a', 'func_a']
call_sandbox("func_a()", scope)

Prove in the second call_ When sandbox, the scope is correctly passed in, there is no report, and func cannot be found_ A also confirms this conclusion. In func_ Get and output globals() and locals() in a:

def func_a():

    inner_scope = dict(globals(), **locals()
    print(inner_scope.keys())
    # ['__builtins__']

You can see in func_a out of scope is normal, but func_ The scope in a is only__ builtins__ The scope has been cleared. The guess is that the caller of the function points to the scope in the sandbox environment. When the scope is returned, the caller is not updated, so the scope outside the function cannot be found in the function. Check the magic method of Python function:

Found one__ globals__ The variable refers to the scope, which is equivalent to the caller of the function. Verify the func in the scope after calling the sandbox service through the following code_ A yes__ globals__ Same as the current scope:

scope["func_a"].__globals__ == globals()  # False

It's really different. Next, try putting Scope ["func_a"]__ globals__ Set it to globals(), and you should be able to run through it.

Optimize scope update logic

Here, the root cause of the problem has been clarified:

-The first and second exec statements are executed in Python services A and B respectively. Func defined in the first exec statement_ The scope of A is service A (func_a. _globals_ = = A)

-After the scope is returned to service B, global_a and func_a is copied to the scope of service B, but func_a.__globals__ It also points to the scope of service a, so func can be called when it appears_ A but in func_ Global not found in a_ a

-Set func_a.__globals__ Set to B to make the code execute correctly in service B

As described in the documentation, the function__ globals__ It is a read-only variable, so it cannot be assigned directly. It needs to be implemented by copying function. The method of defining a copying function is as follows:

import copy

import types
import functools
def copy_func(f, globals=None, module=None):
    if globals is None:
        globals = f.__globals__
    g = types.FunctionType(f.__code__, globals, name=f.__name__,
                           argdefs=f.__defaults__, closure=f.__closure__)
    g = functools.update_wrapper(g, f)
    if module is not None:
        g.__module__ = module
    return g

Update the scope returned after calling the sandbox. If the value in the scope is a function, update its value by copying__ globals__ For scope:

scope = dict(globals(), **locals())

scope = call_sandbox(
"""
global_a = 123
def func_a():
    print(global_a)
"""
, scope)
for k, v in scope:
    if isinstance(v, types.FunctionType):
        scope[k] = copy_func(v, scope, __name__)
call_sandbox("func_a()", scope)

Rerun, two calls_ Sandbox can be executed normally, and the problem is solved.

Reference documents

https://docs.python.org/3/reference/datamodel.html

https://stackoverflow.com/questions/49076566/override-globals-in-function-imported-from-another-module

https://stackoverflow.com/questions/2904274/globals-and-locals-in-python-exec/2906198