### Godot version

v4.1.dev.custom_build [a67d37f7c]

### System information

Windows 10, Vulkan

### Issue description

When changing between scenes that have volumetric fog enabled in the world environment, for example with `change_scene_to_file`, after changing there will be a continuous stream of errors that doesn't seem to stop.
Leaving a copy of one of the errors here so it's findable when searching the issue: 
"RenderingDeviceVulkan::compute_list_bind_uniform_set: Condition "!uniform_set" is true.
  <C++ Source>   drivers\vulkan\rendering_device_vulkan.cpp:7816 @ RenderingDeviceVulkan::compute_list_bind_uniform_set()"

The MRP is the volumetric fog demo, with its main scene duplicated, and the left/right arrow keys being used to change between scenes.

Disabling vol fog in either scene makes the errors go away.

![image](https://github.com/godotengine/godot/assets/115422160/f2c8ef63-f3f3-49ed-8a44-bbee88c9c352)



### Steps to reproduce

Use left/right arrow keys to change scene and back, then error spam should begin.

### Minimal reproduction project

[new hot volumetric_fog bug.zip](https://github.com/godotengine/godot/files/11575612/new.hot.volumetric_fog.bug.zip)

RendererRD::MaterialStorage::store_transform(fog_volume_instance->transform.affine_inverse(), push_constant.transform);
			RD::get_singleton()->compute_list_bind_compute_pipeline(compute_list, shader_data->pipeline);
			RD::get_singleton()->compute_list_bind_uniform_set(compute_list, fog->fog_uniform_set, VolumetricFogShader::FogSet::FOG_SET_UNIFORMS);
			RD::get_singleton()->compute_list_set_push_constant(compute_list, &push_constant, sizeof(VolumetricFogShader::FogPushConstant));
			RD::get_singleton()->compute_list_bind_uniform_set(compute_list, volumetric_fog.base_uniform_set, VolumetricFogShader::FogSet::FOG_SET_BASE);
			if (material->uniform_set.is_valid() && RD::get_singleton()->uniform_set_is_valid(material->uniform_set)) { 
				RD::get_singleton()->compute_list_bind_uniform_set(compute_list, material->uniform_set, VolumetricFogShader::FogSet::FOG_SET_MATERIAL);
				material->set_as_used();
			}
			RD::get_singleton()->compute_list_dispatch_threads(compute_list, kernel_size.x, kernel_size.y, kernel_size.z);
		}
		if (any_uses_time || RendererSceneRenderRD::get_singleton()->environment_get_volumetric_fog_temporal_reprojection(p_settings.env)) {
			RenderingServerDefault::redraw_request();
		}
		RD::get_singleton()->draw_command_end_label();
		RD::get_singleton()->compute_list_end();
	}
	if (!fog->sync_gi_dependent_sets_validity()) {
		
		Vector<RD::Uniform> uniforms;
		Vector<RD::Uniform> copy_uniforms;
		{
			RD::Uniform u;
			u.uniform_type = RD::UNIFORM_TYPE_TEXTURE;
			u.binding = 1;
			if (p_settings.shadow_atlas_depth.is_null()) {
				u.append_id(texture_storage->texture_rd_get_default(RendererRD::TextureStorage::DEFAULT_RD_TEXTURE_BLACK));
			} else {
				u.append_id(p_settings.shadow_atlas_depth);
			}