### Godot version

4.1 beta1

### System information

Windows 10.0.19045 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1060 (NVIDIA; 30.0.15.1403) - Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (8 Threads)

### Issue description

When I change tile source, the view is initialized at a wrong position. After scrolling it for the first time, the position gets re-adjusted.

https://github.com/godotengine/godot/assets/2223172/830cef4a-10f0-4fa5-9172-e7f8015330eb

It wasn't happening before.

### Steps to reproduce

1. Create a TileMap with TileSet
2. Add a sufficiently big tile atlas source
3. Add another source, so that you can switch between the 2
4. Switch to the other source and then to the big one
5. Scroll

### Minimal reproduction project

The texture from the video: https://github.com/godotengine/godot/assets/2223172/7ac78271-84d0-48ab-ae28-45841e2e0880
set_anchor(p_side, p_anchor);
}
void Control::set_anchor(Side p_side, real_t p_anchor, bool p_keep_offset, bool p_push_opposite_anchor) {
	ERR_MAIN_THREAD_GUARD;
	ERR_FAIL_INDEX((int)p_side, 4);
	Rect2 parent_rect = get_parent_anchorable_rect();
	real_t parent_range = (p_side == SIDE_LEFT || p_side == SIDE_RIGHT) ? parent_rect.size.x : parent_rect.size.y;
	real_t previous_pos = data.offset[p_side] + data.anchor[p_side] * parent_range;
	real_t previous_opposite_pos = data.offset[(p_side + 2) % 4] + data.anchor[(p_side + 2) % 4] * parent_range;
	data.anchor[p_side] = p_anchor;
	if (((p_side == SIDE_LEFT || p_side == SIDE_TOP) && data.anchor[p_side] > data.anchor[(p_side + 2) % 4]) ||
			((p_side == SIDE_RIGHT || p_side == SIDE_BOTTOM) && data.anchor[p_side] < data.anchor[(p_side + 2) % 4])) {
		if (p_push_opposite_anchor) {
			data.anchor[(p_side + 2) % 4] = data.anchor[p_side];
		} else {
			data.anchor[p_side] = data.anchor[(p_side + 2) % 4];
		}
	}
	if (!p_keep_offset) {
		data.offset[p_side] = previous_pos - data.anchor[p_side] * parent_range;
		if (p_push_opposite_anchor) {
			data.offset[(p_side + 2) % 4] = previous_opposite_pos - data.anchor[(p_side + 2) % 4] * parent_range;
		}
	}
	if (is_inside_tree()) {
		_size_changed();
	}
	queue_redraw();
}
real_t Control::get_anchor(Side p_side) const {