### Godot version

v4.1.dev.custom_build [a67d37f7c]

### System information

Windows 10, Vulkan

### Issue description

When changing between scenes that have volumetric fog enabled in the world environment, for example with `change_scene_to_file`, after changing there will be a continuous stream of errors that doesn't seem to stop.
Leaving a copy of one of the errors here so it's findable when searching the issue: 
"RenderingDeviceVulkan::compute_list_bind_uniform_set: Condition "!uniform_set" is true.
  <C++ Source>   drivers\vulkan\rendering_device_vulkan.cpp:7816 @ RenderingDeviceVulkan::compute_list_bind_uniform_set()"

The MRP is the volumetric fog demo, with its main scene duplicated, and the left/right arrow keys being used to change between scenes.

Disabling vol fog in either scene makes the errors go away.

![image](https://github.com/godotengine/godot/assets/115422160/f2c8ef63-f3f3-49ed-8a44-bbee88c9c352)



### Steps to reproduce

Use left/right arrow keys to change scene and back, then error spam should begin.

### Minimal reproduction project

[new hot volumetric_fog bug.zip](https://github.com/godotengine/godot/files/11575612/new.hot.volumetric_fog.bug.zip)

const FogVolume *fog_volume = fog_volume_owner.get_or_null(p_fog_volume);
	ERR_FAIL_COND_V(!fog_volume, Vector3());
	return fog_volume->size;
}
bool Fog::FogMaterialData::update_parameters(const HashMap<StringName, Variant> &p_parameters, bool p_uniform_dirty, bool p_textures_dirty) {
	uniform_set_updated = true;
	return update_parameters_uniform_set(p_parameters, p_uniform_dirty, p_textures_dirty, shader_data->uniforms, shader_data->ubo_offsets.ptr(), shader_data->texture_uniforms, shader_data->default_texture_params, shader_data->ubo_size, uniform_set, Fog::get_singleton()->volumetric_fog.shader.version_get_shader(shader_data->version, 0), VolumetricFogShader::FogSet::FOG_SET_MATERIAL, true);
}
Fog::FogMaterialData::~FogMaterialData() {
	free_parameters_uniform_set(uniform_set);
}
RendererRD::MaterialStorage::ShaderData *Fog::_create_fog_shader_func() {
	FogShaderData *shader_data = memnew(FogShaderData);
	return shader_data;
}
RendererRD::MaterialStorage::ShaderData *Fog::_create_fog_shader_funcs() {
	return Fog::get_singleton()->_create_fog_shader_func();
};
RendererRD::MaterialStorage::MaterialData *Fog::_create_fog_material_func(FogShaderData *p_shader) {
	FogMaterialData *material_data = memnew(FogMaterialData);
	material_data->shader_data = p_shader;
	
	return material_data;
}
RendererRD::MaterialStorage::MaterialData *Fog::_create_fog_material_funcs(RendererRD::MaterialStorage::ShaderData *p_shader) {