### Godot version

v4.1.rc1.mono.official [1f9e540f1]

### System information

Godot v4.1.rc1.mono - Windows 10.0.19045 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1660 Ti with Max-Q Design (NVIDIA; 31.0.15.3179) - Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 Threads)

### Issue description

When NavigationAgent3D is instantiated at runtime and added to the scene tree, invisible/impassable obstacle is added at the global origin position (0, 0, 0).

### Steps to reproduce

1. Run attached MRP
2. Click with `left mouse button` at the world origin to make agent walk
3. Click `space` on your keyboard to instantiate NavigationAgent3D
4. Click left mouse button to walk through world origin position again
5. Player gets stuck on obstacle at the world origin position

https://github.com/godotengine/godot/assets/8630984/fc9b32fb-25cb-46d6-9a98-ff299e2c0b1c



### Minimal reproduction project

[MRP.zip](https://github.com/godotengine/godot/files/11910363/MRP.zip)

ERR_FAIL_NULL(NavigationServer3D::get_singleton());
	NavigationServer3D::get_singleton()->free(agent);
	agent = RID(); 
#ifdef DEBUG_ENABLED
	NavigationServer3D::get_singleton()->disconnect(SNAME("navigation_debug_changed"), callable_mp(this, &NavigationAgent3D::_navigation_debug_changed));
	ERR_FAIL_NULL(RenderingServer::get_singleton());
	if (debug_path_instance.is_valid()) {
		RenderingServer::get_singleton()->free(debug_path_instance);
	}
	if (debug_path_mesh.is_valid()) {
		RenderingServer::get_singleton()->free(debug_path_mesh->get_rid());
	}
#endif 
}
void NavigationAgent3D::set_avoidance_enabled(bool p_enabled) {
	if (avoidance_enabled == p_enabled) {
		return;
	}
	avoidance_enabled = p_enabled;
	if (avoidance_enabled) {
		NavigationServer3D::get_singleton()->agent_set_avoidance_enabled(agent, true);
		NavigationServer3D::get_singleton()->agent_set_avoidance_callback(agent, callable_mp(this, &NavigationAgent3D::_avoidance_done));
	} else {
		NavigationServer3D::get_singleton()->agent_set_avoidance_enabled(agent, false);
		NavigationServer3D::get_singleton()->agent_set_avoidance_callback(agent, Callable());
	}
}
bool NavigationAgent3D::get_avoidance_enabled() const {