Have you tried connecting and disconnecting the signal? This Godot QA answer has some example code that shows how to connect and disconnect signals using GDScript. You would just need to change the code so that the special key connects/disconnects the signal instead of is_inside_tree() like in the example in the QA answer.
Or if you want you can just ignore the signal unless the special key is pressed. It requires some additional processing, and is perhaps more of a workaround than anything else, but it is easy enough to use.
In theory the code would look something like this (untested):
var special_key_down = false;
func _process(delta):
if (Input.is_action_pressed("special_key")):
special_key_down = true;
else:
special_key_down = false;
func on_area_entered(area):
if (special_key_down == false):
return;
# Insert signal related code here. For example
print ("The special key is pressed!")
Hopefully this helps :smile: