### Godot version

4.1 beta1

### System information

Windows 10.0.19045 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1060 (NVIDIA; 30.0.15.1403) - Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (8 Threads)

### Issue description

When I change tile source, the view is initialized at a wrong position. After scrolling it for the first time, the position gets re-adjusted.

https://github.com/godotengine/godot/assets/2223172/830cef4a-10f0-4fa5-9172-e7f8015330eb

It wasn't happening before.

### Steps to reproduce

1. Create a TileMap with TileSet
2. Add a sufficiently big tile atlas source
3. Add another source, so that you can switch between the 2
4. Switch to the other source and then to the big one
5. Scroll

### Minimal reproduction project

The texture from the video: https://github.com/godotengine/godot/assets/2223172/7ac78271-84d0-48ab-ae28-45841e2e0880
ERR_READ_THREAD_GUARD_V(Size2());
	return data.custom_minimum_size;
}
void Control::_update_minimum_size_cache() {
	Size2 minsize = get_minimum_size();
	minsize.x = MAX(minsize.x, data.custom_minimum_size.x);
	minsize.y = MAX(minsize.y, data.custom_minimum_size.y);
	bool size_changed = false;
	if (data.minimum_size_cache != minsize) {
		size_changed = true;
	}
	data.minimum_size_cache = minsize;
	data.minimum_size_valid = true;
	if (size_changed) {
		update_minimum_size();
	}
}
Size2 Control::get_combined_minimum_size() const {
	ERR_READ_THREAD_GUARD_V(Size2());
	if (!data.minimum_size_valid) {
		const_cast<Control *>(this)->_update_minimum_size_cache();
	}
	return data.minimum_size_cache;
}
void Control::_size_changed() {