[ci] pull main branch before diffing (#68983)

we tried to generate a full diff against main in
ec9d80e but it resulted in wrong diffs.

It seems that the issue was that 'main' was not
updated after agent restart and diff main...HEAD kept growing.

Not enabling diff main...HEAD just yet and will check logs for new PRs
first.

Author: metaflow

.ci/generate-buildkite-pipeline-premerge

@@ -23,6 +23,16 @@ set -o pipefail
 # Environment variables script works with:
 # List of files affected by this commit
 : ${MODIFIED_FILES:=$(git diff --name-only HEAD~1)}
+# Fetch origin/main to have an up to date merge base for main...HEAD diff.
+git fetch origin main:main
+echo "files modified HEAD~1" >&2
+git --no-pager diff --name-only HEAD~1 >&2
+echo "files modified main...HEAD" >&2
+git --no-pager diff --name-only main...HEAD | head -n 10 >&2
+merge_base=$(git merge-base main HEAD)
+echo "merge base with main $merge_base" >&2
+echo "git log" >&2
+git --no-pager log --oneline --abbrev-commit -n 5 >&2
 # Filter rules for generic windows tests
 : ${WINDOWS_AGENTS:='{"queue": "windows"}'}
 # Filter rules for generic linux tests