Skip to content

Commit e038c54

Browse files
authored
[LTO][Pipelines] Add 0 hot-caller threshold for SamplePGO + FullLTO (#135152)
If a hot callsite function is not inlined in the 1st build, inlining the hot callsite in pre-link stage of SPGO 2nd build may lead to Function Sample not found in profile file in link stage. It will miss some profile info. ThinLTO has already considered and dealed with it by setting HotCallSiteThreshold to 0 to stop the inline. This patch just adds the same processing for FullLTO.
1 parent 7778a19 commit e038c54

File tree

2 files changed

+12
-9
lines changed

2 files changed

+12
-9
lines changed

llvm/lib/Passes/PassBuilderPipelines.cpp

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -926,14 +926,14 @@ PassBuilder::buildInlinerPipeline(OptimizationLevel Level,
926926
IP = getInlineParamsFromOptLevel(Level);
927927
else
928928
IP = getInlineParams(PTO.InlinerThreshold);
929-
// For PreLinkThinLTO + SamplePGO, set hot-caller threshold to 0 to
930-
// disable hot callsite inline (as much as possible [1]) because it makes
929+
// For PreLinkThinLTO + SamplePGO or PreLinkFullLTO + SamplePGO,
930+
// set hot-caller threshold to 0 to disable hot
931+
// callsite inline (as much as possible [1]) because it makes
931932
// profile annotation in the backend inaccurate.
932933
//
933934
// [1] Note the cost of a function could be below zero due to erased
934935
// prologue / epilogue.
935-
if (Phase == ThinOrFullLTOPhase::ThinLTOPreLink && PGOOpt &&
936-
PGOOpt->Action == PGOOptions::SampleUse)
936+
if (isLTOPreLink(Phase) && PGOOpt && PGOOpt->Action == PGOOptions::SampleUse)
937937
IP.HotCallSiteThreshold = 0;
938938

939939
if (PGOOpt)
@@ -1023,14 +1023,14 @@ PassBuilder::buildModuleInlinerPipeline(OptimizationLevel Level,
10231023
ModulePassManager MPM;
10241024

10251025
InlineParams IP = getInlineParamsFromOptLevel(Level);
1026-
// For PreLinkThinLTO + SamplePGO, set hot-caller threshold to 0 to
1027-
// disable hot callsite inline (as much as possible [1]) because it makes
1026+
// For PreLinkThinLTO + SamplePGO or PreLinkFullLTO + SamplePGO,
1027+
// set hot-caller threshold to 0 to disable hot
1028+
// callsite inline (as much as possible [1]) because it makes
10281029
// profile annotation in the backend inaccurate.
10291030
//
10301031
// [1] Note the cost of a function could be below zero due to erased
10311032
// prologue / epilogue.
1032-
if (Phase == ThinOrFullLTOPhase::ThinLTOPreLink && PGOOpt &&
1033-
PGOOpt->Action == PGOOptions::SampleUse)
1033+
if (isLTOPreLink(Phase) && PGOOpt && PGOOpt->Action == PGOOptions::SampleUse)
10341034
IP.HotCallSiteThreshold = 0;
10351035

10361036
if (PGOOpt)

llvm/test/Other/new-pm-thinlto-prelink-samplepgo-inline-threshold.ll renamed to llvm/test/Other/new-pm-lto-prelink-samplepgo-inline-threshold.ll

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
; Tests that hot callsite threshold is set to 0 artifically for thinlto-prelink pipeline.
1+
; Tests that hot callsite threshold is set to 0 artifically for thinlto-prelink and lto-pre-link pipeline.
22
;
33
; Function `sum` is annotated with inline cost -1 and function `sum1` is
44
; annotated with inline cost 0, by function attribute `function-inline-cost`.
@@ -12,6 +12,9 @@
1212

1313
; RUN: opt < %s -pass-remarks=inline -pass-remarks-missed=inline -passes='thinlto-pre-link<O2>' -pgo-kind=pgo-sample-use-pipeline -sample-profile-file=%S/Inputs/new-pm-thinlto-prelink-samplepgo-inline-threshold.prof -S 2>&1 | FileCheck %s -check-prefix=REMARK
1414

15+
; RUN: opt < %s -pass-remarks=inline -pass-remarks-missed=inline -passes='lto-pre-link<O2>' -pgo-kind=pgo-sample-use-pipeline -sample-profile-file=%S/Inputs/new-pm-thinlto-prelink-samplepgo-inline-threshold.prof -S | FileCheck %s
16+
17+
; RUN: opt < %s -pass-remarks=inline -pass-remarks-missed=inline -passes='lto-pre-link<O2>' -pgo-kind=pgo-sample-use-pipeline -sample-profile-file=%S/Inputs/new-pm-thinlto-prelink-samplepgo-inline-threshold.prof -S 2>&1 | FileCheck %s -check-prefix=REMARK
1518
; Original C++ test case
1619
;
1720
; #include <stdio.h>

0 commit comments

Comments
 (0)