[lld][WebAssembly] Implement various thinlto flags #114327
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@llvm/pr-subscribers-lld-wasm Author: Sam Clegg (sbc100) ChangesThe changes in this PR (both in the code and the tests) are largely copied directly from the ELF linker. Parial fix for #79604. Full diff: https://github.com/llvm/llvm-project/pull/114327.diff 8 Files Affected:
diff --git a/lld/test/wasm/lto/obj-path.ll b/lld/test/wasm/lto/obj-path.ll
new file mode 100644
index 00000000000000..3a8dba15e9c1d8
--- /dev/null
+++ b/lld/test/wasm/lto/obj-path.ll
@@ -0,0 +1,94 @@
+;; Copied from testr/ELF/lto/obj-path.ll
+;; Test --lto-obj-path= for regular LTO.
+
+; RUN: rm -rf %t && split-file %s %t && cd %t
+; RUN: mkdir d
+; RUN: opt 1.ll -o 1.bc
+; RUN: opt 2.ll -o d/2.bc
+
+; RUN: rm -f objpath.o
+; RUN: wasm-ld --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o 3
+; RUN: llvm-nm 3 | FileCheck %s --check-prefix=NM
+; RUN: llvm-objdump -d objpath.o | FileCheck %s
+; RUN: ls 3* objpath* | count 2
+
+; RUN: rm -f 3 objpath.o
+; RUN: wasm-ld --thinlto-index-only=3.txt --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o 3
+; RUN: llvm-objdump -d objpath.o | FileCheck %s
+; RUN: not ls 3
+
+; NM: T f
+; NM: T g
+
+; CHECK: file format wasm
+; CHECK: <f>:
+; CHECK: <g>:
+
+;; Test --lto-obj-path= for ThinLTO.
+; RUN: opt -module-summary 1.ll -o 1.bc
+; RUN: opt -module-summary 2.ll -o d/2.bc
+
+; RUN: wasm-ld --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o 3
+; RUN: llvm-nm 3 | FileCheck %s --check-prefix=NM3
+; RUN: llvm-objdump -d objpath.o1 | FileCheck %s --check-prefix=CHECK1
+; RUN: llvm-objdump -d objpath.o2 | FileCheck %s --check-prefix=CHECK2
+
+; NM3: T f
+; NM3-NEXT: T g
+
+; CHECK1: file format wasm
+; CHECK1-EMPTY:
+; CHECK1-NEXT: Disassembly of section CODE:
+; CHECK1: <f>:
+; CHECK1-EMPTY:
+; CHECK1-NEXT: end
+; CHECK1-NOT: {{.}}
+
+; CHECK2: file format wasm
+; CHECK2-EMPTY:
+; CHECK2-NEXT: Disassembly of section CODE:
+; CHECK2: <g>:
+; CHECK2-EMPTY:
+; CHECK2-NEXT: end
+; CHECK2-NOT: {{.}}
+
+;; With --thinlto-index-only, --lto-obj-path= creates just one file.
+; RUN: rm -f objpath.o objpath.o1 objpath.o2
+; RUN: wasm-ld --thinlto-index-only --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o /dev/null
+; RUN: llvm-objdump -d objpath.o | FileCheck %s --check-prefix=EMPTY
+; RUN: not ls objpath.o1
+; RUN: not ls objpath.o2
+
+;; Ensure lld emits empty combined module if specific obj-path.
+; RUN: mkdir obj
+; RUN: wasm-ld --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o obj/out --save-temps
+; RUN: ls obj/out.lto.o out.lto.1.o d/out.lto.2.o
+
+;; Ensure lld does not emit empty combined module by default.
+; RUN: rm -fr obj && mkdir obj
+; RUN: wasm-ld -shared 1.bc d/2.bc -o obj/out --save-temps
+; RUN: not test -e obj/out.lto.o
+
+; EMPTY: file format wasm
+; EMPTY-NOT: {{.}}
+
+;--- 1.ll
+target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
+target triple = "wasm32-unknown-unknown"
+
+declare void @g(...)
+
+define void @f() {
+entry:
+ call void (...) @g()
+ ret void
+}
+
+;--- 2.ll
+target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
+target triple = "wasm32-unknown-unknown"
+
+define void @g() {
+entry:
+ ret void
+}
diff --git a/lld/test/wasm/lto/parallel.ll b/lld/test/wasm/lto/parallel.ll
index a9e8b653d11a35..507fea6ee8558b 100644
--- a/lld/test/wasm/lto/parallel.ll
+++ b/lld/test/wasm/lto/parallel.ll
@@ -1,8 +1,8 @@
-; RUN: llvm-as -o %t.bc %s
-; RUN: rm -f %t.lto.o %t1.lto.o
-; RUN: wasm-ld --lto-partitions=2 -save-temps -o %t %t.bc -r
-; RUN: llvm-nm %t.lto.o | FileCheck --check-prefix=CHECK0 %s
-; RUN: llvm-nm %t1.lto.o | FileCheck --check-prefix=CHECK1 %s
+; RUN: rm -rf %t && mkdir %t && cd %t
+; RUN: llvm-as -o a.bc %s
+; RUN: wasm-ld --lto-partitions=2 -save-temps -o out a.bc -r
+; RUN: llvm-nm out.lto.o | FileCheck --check-prefix=CHECK0 %s
+; RUN: llvm-nm out.lto.1.o | FileCheck --check-prefix=CHECK1 %s
target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
target triple = "wasm32-unknown-unknown-wasm"
diff --git a/lld/test/wasm/lto/thinlto.ll b/lld/test/wasm/lto/thinlto.ll
index c7dd73adcd4b76..df539cc38c83a0 100644
--- a/lld/test/wasm/lto/thinlto.ll
+++ b/lld/test/wasm/lto/thinlto.ll
@@ -1,53 +1,56 @@
; Basic ThinLTO tests.
-; RUN: opt -module-summary %s -o %t1.o
-; RUN: opt -module-summary %p/Inputs/thinlto.ll -o %t2.o
+; RUN: rm -rf %t && mkdir %t && cd %t
+; RUN: mkdir d e
+
+; RUN: opt -module-summary %s -o a.o
+; RUN: opt -module-summary %p/Inputs/thinlto.ll -o d/b.o
; First force single-threaded mode
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --thinlto-jobs=1 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --thinlto-jobs=1 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; Next force multi-threaded mode
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --thinlto-jobs=2 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --thinlto-jobs=2 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
;; --thinlto-jobs= defaults to --threads=.
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --threads=2 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --threads=2 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
;; --thinlto-jobs= overrides --threads=.
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --threads=1 --thinlto-jobs=2 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --threads=1 --thinlto-jobs=2 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; Test with all threads, on all cores, on all CPU sockets
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --thinlto-jobs=all %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --thinlto-jobs=all a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; Test with many more threads than the system has
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --thinlto-jobs=100 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --thinlto-jobs=100 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; Test with a bad value
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: not wasm-ld -r -save-temps --thinlto-jobs=foo %t1.o %t2.o -o %t3 2>&1 | FileCheck %s --check-prefix=BAD-JOBS
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: not wasm-ld -r -save-temps --thinlto-jobs=foo a.o d/b.o -o e/out 2>&1 | FileCheck %s --check-prefix=BAD-JOBS
; BAD-JOBS: error: --thinlto-jobs: invalid job count: foo
; Check without --thinlto-jobs (which currently defaults to heavyweight_hardware_concurrency, meanning one thread per hardware core -- not SMT)
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; NM1: T f
; NM2: T g
diff --git a/lld/wasm/Config.h b/lld/wasm/Config.h
index 05a547ff9278a1..7be1927eb22421 100644
--- a/lld/wasm/Config.h
+++ b/lld/wasm/Config.h
@@ -79,6 +79,8 @@ struct Configuration {
// Because dyamanic linking under Wasm is still experimental we default to
// static linking
bool isStatic = true;
+ bool thinLTOEmitIndexFiles;
+ bool thinLTOIndexOnly;
bool trace;
uint64_t globalBase;
uint64_t initialHeap;
@@ -95,16 +97,18 @@ struct Configuration {
unsigned ltoo;
llvm::CodeGenOptLevel ltoCgo;
unsigned optimize;
- llvm::StringRef thinLTOJobs;
bool ltoDebugPassManager;
UnresolvedPolicy unresolvedSymbols;
BuildIdKind buildId = BuildIdKind::None;
llvm::StringRef entry;
+ llvm::StringRef ltoObjPath;
llvm::StringRef mapFile;
llvm::StringRef outputFile;
llvm::StringRef soName;
llvm::StringRef thinLTOCacheDir;
+ llvm::StringRef thinLTOJobs;
+ llvm::StringRef thinLTOIndexOnlyArg;
llvm::StringRef whyExtract;
llvm::StringSet<> allowUndefinedSymbols;
diff --git a/lld/wasm/Driver.cpp b/lld/wasm/Driver.cpp
index 9a27fc90457f08..9076a320a48e39 100644
--- a/lld/wasm/Driver.cpp
+++ b/lld/wasm/Driver.cpp
@@ -542,6 +542,7 @@ static void readConfigs(opt::InputArgList &args) {
else
error("invalid codegen optimization level for LTO: " + Twine(ltoCgo));
config->ltoPartitions = args::getInteger(args, OPT_lto_partitions, 1);
+ config->ltoObjPath = args.getLastArgValue(OPT_lto_obj_path_eq);
config->ltoDebugPassManager = args.hasArg(OPT_lto_debug_pass_manager);
config->mapFile = args.getLastArgValue(OPT_Map);
config->optimize = args::getInteger(args, OPT_O, 1);
@@ -569,6 +570,12 @@ static void readConfigs(opt::InputArgList &args) {
config->thinLTOCachePolicy = CHECK(
parseCachePruningPolicy(args.getLastArgValue(OPT_thinlto_cache_policy)),
"--thinlto-cache-policy: invalid cache policy");
+ config->thinLTOEmitIndexFiles = args.hasArg(OPT_thinlto_emit_index_files) ||
+ args.hasArg(OPT_thinlto_index_only) ||
+ args.hasArg(OPT_thinlto_index_only_eq);
+ config->thinLTOIndexOnly = args.hasArg(OPT_thinlto_index_only) ||
+ args.hasArg(OPT_thinlto_index_only_eq);
+ config->thinLTOIndexOnlyArg = args.getLastArgValue(OPT_thinlto_index_only_eq);
config->unresolvedSymbols = getUnresolvedSymbolPolicy(args);
config->whyExtract = args.getLastArgValue(OPT_why_extract);
errorHandler().verbose = args.hasArg(OPT_verbose);
@@ -1379,6 +1386,10 @@ void LinkerDriver::linkerMain(ArrayRef<const char *> argsArr) {
writeWhyExtract();
+ // Bail out if normal linked output is skipped due to LTO.
+ if (config->thinLTOIndexOnly)
+ return;
+
createOptionalSymbols();
// Resolve any variant symbols that were created due to signature
diff --git a/lld/wasm/LTO.cpp b/lld/wasm/LTO.cpp
index e523f0f6171535..c03483c2200689 100644
--- a/lld/wasm/LTO.cpp
+++ b/lld/wasm/LTO.cpp
@@ -11,7 +11,9 @@
#include "InputFiles.h"
#include "Symbols.h"
#include "lld/Common/Args.h"
+#include "lld/Common/CommonLinkerContext.h"
#include "lld/Common/ErrorHandler.h"
+#include "lld/Common/Filesystem.h"
#include "lld/Common/Strings.h"
#include "lld/Common/TargetOptionsCommandFlags.h"
#include "llvm/ADT/STLExtras.h"
@@ -27,6 +29,7 @@
#include "llvm/Support/Error.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
#include "llvm/Support/raw_ostream.h"
#include <algorithm>
#include <cstddef>
@@ -36,9 +39,10 @@
#include <vector>
using namespace llvm;
+using namespace lld::wasm;
+using namespace lld;
-namespace lld::wasm {
-static std::unique_ptr<lto::LTO> createLTO() {
+static lto::Config createConfig() {
lto::Config c;
c.Options = initTargetOptionsFromCodeGenFlags();
@@ -52,6 +56,7 @@ static std::unique_ptr<lto::LTO> createLTO() {
c.MAttrs = getMAttrs();
c.CGOptLevel = config->ltoCgo;
c.DebugPassManager = config->ltoDebugPassManager;
+ c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
if (config->relocatable)
c.RelocModel = std::nullopt;
@@ -63,13 +68,30 @@ static std::unique_ptr<lto::LTO> createLTO() {
if (config->saveTemps)
checkError(c.addSaveTemps(config->outputFile.str() + ".",
/*UseInputModulePath*/ true));
- lto::ThinBackend backend = lto::createInProcessThinBackend(
- llvm::heavyweight_hardware_concurrency(config->thinLTOJobs));
- return std::make_unique<lto::LTO>(std::move(c), backend,
- config->ltoPartitions);
+ return c;
}
-BitcodeCompiler::BitcodeCompiler() : ltoObj(createLTO()) {}
+namespace lld::wasm {
+
+BitcodeCompiler::BitcodeCompiler() {
+ // Initialize indexFile.
+ if (!config->thinLTOIndexOnlyArg.empty())
+ indexFile = openFile(config->thinLTOIndexOnlyArg);
+
+ // Initialize ltoObj.
+ lto::ThinBackend backend;
+ auto onIndexWrite = [&](StringRef s) { thinIndices.erase(s); };
+ if (config->thinLTOIndexOnly) {
+ backend = lto::createWriteIndexesThinBackend(
+ llvm::hardware_concurrency(config->thinLTOJobs),
+ "", "", "", false, indexFile.get(), onIndexWrite);
+ } else {
+ backend = lto::createInProcessThinBackend(
+ llvm::heavyweight_hardware_concurrency(config->thinLTOJobs));
+ }
+ ltoObj =
+ std::make_unique<lto::LTO>(createConfig(), backend, config->ltoPartitions);
+}
BitcodeCompiler::~BitcodeCompiler() = default;
@@ -90,6 +112,9 @@ void BitcodeCompiler::add(BitcodeFile &f) {
ArrayRef<Symbol *> syms = f.getSymbols();
std::vector<lto::SymbolResolution> resols(syms.size());
+ if (config->thinLTOEmitIndexFiles)
+ thinIndices.insert(obj.getName());
+
// Provide a resolution to the LTO API for each symbol.
for (const lto::InputFile::Symbol &objSym : obj.symbols()) {
Symbol *sym = syms[symNum];
@@ -136,25 +161,66 @@ std::vector<StringRef> BitcodeCompiler::compile() {
checkError(ltoObj->run(
[&](size_t task, const Twine &moduleName) {
+ buf[task].first = moduleName.str();
return std::make_unique<CachedFileStream>(
- std::make_unique<raw_svector_ostream>(buf[task]));
+ std::make_unique<raw_svector_ostream>(buf[task].second));
},
cache));
if (!config->thinLTOCacheDir.empty())
pruneCache(config->thinLTOCacheDir, config->thinLTOCachePolicy, files);
+ if (config->thinLTOIndexOnly) {
+ if (!config->ltoObjPath.empty())
+ saveBuffer(buf[0].second, config->ltoObjPath);
+
+ // ThinLTO with index only option is required to generate only the index
+ // files. After that, we exit from linker and ThinLTO backend runs in a
+ // distributed environment.
+ if (indexFile)
+ indexFile->close();
+ return {};
+ }
+
std::vector<StringRef> ret;
for (unsigned i = 0; i != maxTasks; ++i) {
- if (buf[i].empty())
+ StringRef objBuf = buf[i].second;
+ StringRef bitcodeFilePath = buf[i].first;
+ if (objBuf.empty())
+ continue;
+ ret.emplace_back(objBuf.data(), objBuf.size());
+ if (!config->saveTemps)
continue;
- if (config->saveTemps) {
- if (i == 0)
- saveBuffer(buf[i], config->outputFile + ".lto.o");
- else
- saveBuffer(buf[i], config->outputFile + Twine(i) + ".lto.o");
+
+ // If the input bitcode file is path/to/x.o and -o specifies a.out, the
+ // corresponding native relocatable file path will look like:
+ // path/to/a.out.lto.x.o.
+ StringRef ltoObjName;
+ if (bitcodeFilePath == "ld-temp.o") {
+ ltoObjName =
+ saver().save(Twine(config->outputFile) + ".lto" +
+ (i == 0 ? Twine("") : Twine('.') + Twine(i)) + ".o");
+ } else {
+ StringRef directory = sys::path::parent_path(bitcodeFilePath);
+ // For an archive member, which has an identifier like "d/a.a(coll.o at
+ // 8)" (see BitcodeFile::BitcodeFile), use the filename; otherwise, use
+ // the stem (d/a.o => a).
+ StringRef baseName = bitcodeFilePath.ends_with(")")
+ ? sys::path::filename(bitcodeFilePath) : sys::path::stem(bitcodeFilePath);
+ StringRef outputFileBaseName = sys::path::filename(config->outputFile);
+ SmallString<256> path;
+ sys::path::append(path, directory,
+ outputFileBaseName + ".lto." + baseName + ".o");
+ sys::path::remove_dots(path, true);
+ ltoObjName = saver().save(path.str());
}
- ret.emplace_back(buf[i].data(), buf[i].size());
+ saveBuffer(objBuf, ltoObjName);
+ }
+
+ if (!config->ltoObjPath.empty()) {
+ saveBuffer(buf[0].second, config->ltoObjPath);
+ for (unsigned i = 1; i != maxTasks; ++i)
+ saveBuffer(buf[i].second, config->ltoObjPath + Twine(i));
}
for (std::unique_ptr<MemoryBuffer> &file : files)
diff --git a/lld/wasm/LTO.h b/lld/wasm/LTO.h
index bb57c6651394b7..c61faf82b8fe6d 100644
--- a/lld/wasm/LTO.h
+++ b/lld/wasm/LTO.h
@@ -21,7 +21,9 @@
#define LLD_WASM_LTO_H
#include "lld/Common/LLVM.h"
+#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/raw_ostream.h"
#include "Writer.h"
#include <memory>
#include <vector>
@@ -47,8 +49,11 @@ class BitcodeCompiler {
private:
std::unique_ptr<llvm::lto::LTO> ltoObj;
- std::vector<SmallString<0>> buf;
+ // An array of (module name, native relocatable file content) pairs.
+ SmallVector<std::pair<std::string, SmallString<0>>, 0> buf;
std::vector<std::unique_ptr<MemoryBuffer>> files;
+ std::unique_ptr<llvm::raw_fd_ostream> indexFile;
+ llvm::DenseSet<StringRef> thinIndices;
};
} // namespace lld::wasm
diff --git a/lld/wasm/Options.td b/lld/wasm/Options.td
index cff29e709a1a09..2ffe79f970f1fe 100644
--- a/lld/wasm/Options.td
+++ b/lld/wasm/Options.td
@@ -297,11 +297,15 @@ def lto_CGO: JJ<"lto-CGO">, MetaVarName<"<cgopt-level>">,
HelpText<"Codegen optimization level for LTO">;
def lto_partitions: JJ<"lto-partitions=">,
HelpText<"Number of LTO codegen partitions">;
+def lto_obj_path_eq: JJ<"lto-obj-path=">;
def disable_verify: F<"disable-verify">;
def save_temps: F<"save-temps">, HelpText<"Save intermediate LTO compilation results">;
def thinlto_cache_dir: JJ<"thinlto-cache-dir=">,
HelpText<"Path to ThinLTO cached object file directory">;
defm thinlto_cache_policy: EEq<"thinlto-cache-policy", "Pruning policy for the ThinLTO cache">;
+def thinlto_emit_index_files: FF<"thinlto-emit-index-files">;
+def thinlto_index_only: FF<"thinlto-index-only">;
+def thinlto_index_only_eq: JJ<"thinlto-index-only=">;
def thinlto_jobs: JJ<"thinlto-jobs=">,
HelpText<"Number of ThinLTO jobs. Default to --threads=">;
def lto_debug_pass_manager: FF<"lto-debug-pass-manager">,
|
|
@llvm/pr-subscribers-lld Author: Sam Clegg (sbc100) ChangesThe changes in this PR (both in the code and the tests) are largely copied directly from the ELF linker. Parial fix for #79604. Full diff: https://github.com/llvm/llvm-project/pull/114327.diff 8 Files Affected:
diff --git a/lld/test/wasm/lto/obj-path.ll b/lld/test/wasm/lto/obj-path.ll
new file mode 100644
index 00000000000000..3a8dba15e9c1d8
--- /dev/null
+++ b/lld/test/wasm/lto/obj-path.ll
@@ -0,0 +1,94 @@
+;; Copied from testr/ELF/lto/obj-path.ll
+;; Test --lto-obj-path= for regular LTO.
+
+; RUN: rm -rf %t && split-file %s %t && cd %t
+; RUN: mkdir d
+; RUN: opt 1.ll -o 1.bc
+; RUN: opt 2.ll -o d/2.bc
+
+; RUN: rm -f objpath.o
+; RUN: wasm-ld --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o 3
+; RUN: llvm-nm 3 | FileCheck %s --check-prefix=NM
+; RUN: llvm-objdump -d objpath.o | FileCheck %s
+; RUN: ls 3* objpath* | count 2
+
+; RUN: rm -f 3 objpath.o
+; RUN: wasm-ld --thinlto-index-only=3.txt --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o 3
+; RUN: llvm-objdump -d objpath.o | FileCheck %s
+; RUN: not ls 3
+
+; NM: T f
+; NM: T g
+
+; CHECK: file format wasm
+; CHECK: <f>:
+; CHECK: <g>:
+
+;; Test --lto-obj-path= for ThinLTO.
+; RUN: opt -module-summary 1.ll -o 1.bc
+; RUN: opt -module-summary 2.ll -o d/2.bc
+
+; RUN: wasm-ld --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o 3
+; RUN: llvm-nm 3 | FileCheck %s --check-prefix=NM3
+; RUN: llvm-objdump -d objpath.o1 | FileCheck %s --check-prefix=CHECK1
+; RUN: llvm-objdump -d objpath.o2 | FileCheck %s --check-prefix=CHECK2
+
+; NM3: T f
+; NM3-NEXT: T g
+
+; CHECK1: file format wasm
+; CHECK1-EMPTY:
+; CHECK1-NEXT: Disassembly of section CODE:
+; CHECK1: <f>:
+; CHECK1-EMPTY:
+; CHECK1-NEXT: end
+; CHECK1-NOT: {{.}}
+
+; CHECK2: file format wasm
+; CHECK2-EMPTY:
+; CHECK2-NEXT: Disassembly of section CODE:
+; CHECK2: <g>:
+; CHECK2-EMPTY:
+; CHECK2-NEXT: end
+; CHECK2-NOT: {{.}}
+
+;; With --thinlto-index-only, --lto-obj-path= creates just one file.
+; RUN: rm -f objpath.o objpath.o1 objpath.o2
+; RUN: wasm-ld --thinlto-index-only --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o /dev/null
+; RUN: llvm-objdump -d objpath.o | FileCheck %s --check-prefix=EMPTY
+; RUN: not ls objpath.o1
+; RUN: not ls objpath.o2
+
+;; Ensure lld emits empty combined module if specific obj-path.
+; RUN: mkdir obj
+; RUN: wasm-ld --lto-obj-path=objpath.o -shared 1.bc d/2.bc -o obj/out --save-temps
+; RUN: ls obj/out.lto.o out.lto.1.o d/out.lto.2.o
+
+;; Ensure lld does not emit empty combined module by default.
+; RUN: rm -fr obj && mkdir obj
+; RUN: wasm-ld -shared 1.bc d/2.bc -o obj/out --save-temps
+; RUN: not test -e obj/out.lto.o
+
+; EMPTY: file format wasm
+; EMPTY-NOT: {{.}}
+
+;--- 1.ll
+target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
+target triple = "wasm32-unknown-unknown"
+
+declare void @g(...)
+
+define void @f() {
+entry:
+ call void (...) @g()
+ ret void
+}
+
+;--- 2.ll
+target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
+target triple = "wasm32-unknown-unknown"
+
+define void @g() {
+entry:
+ ret void
+}
diff --git a/lld/test/wasm/lto/parallel.ll b/lld/test/wasm/lto/parallel.ll
index a9e8b653d11a35..507fea6ee8558b 100644
--- a/lld/test/wasm/lto/parallel.ll
+++ b/lld/test/wasm/lto/parallel.ll
@@ -1,8 +1,8 @@
-; RUN: llvm-as -o %t.bc %s
-; RUN: rm -f %t.lto.o %t1.lto.o
-; RUN: wasm-ld --lto-partitions=2 -save-temps -o %t %t.bc -r
-; RUN: llvm-nm %t.lto.o | FileCheck --check-prefix=CHECK0 %s
-; RUN: llvm-nm %t1.lto.o | FileCheck --check-prefix=CHECK1 %s
+; RUN: rm -rf %t && mkdir %t && cd %t
+; RUN: llvm-as -o a.bc %s
+; RUN: wasm-ld --lto-partitions=2 -save-temps -o out a.bc -r
+; RUN: llvm-nm out.lto.o | FileCheck --check-prefix=CHECK0 %s
+; RUN: llvm-nm out.lto.1.o | FileCheck --check-prefix=CHECK1 %s
target datalayout = "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20"
target triple = "wasm32-unknown-unknown-wasm"
diff --git a/lld/test/wasm/lto/thinlto.ll b/lld/test/wasm/lto/thinlto.ll
index c7dd73adcd4b76..df539cc38c83a0 100644
--- a/lld/test/wasm/lto/thinlto.ll
+++ b/lld/test/wasm/lto/thinlto.ll
@@ -1,53 +1,56 @@
; Basic ThinLTO tests.
-; RUN: opt -module-summary %s -o %t1.o
-; RUN: opt -module-summary %p/Inputs/thinlto.ll -o %t2.o
+; RUN: rm -rf %t && mkdir %t && cd %t
+; RUN: mkdir d e
+
+; RUN: opt -module-summary %s -o a.o
+; RUN: opt -module-summary %p/Inputs/thinlto.ll -o d/b.o
; First force single-threaded mode
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --thinlto-jobs=1 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --thinlto-jobs=1 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; Next force multi-threaded mode
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --thinlto-jobs=2 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --thinlto-jobs=2 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
;; --thinlto-jobs= defaults to --threads=.
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --threads=2 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --threads=2 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
;; --thinlto-jobs= overrides --threads=.
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --threads=1 --thinlto-jobs=2 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --threads=1 --thinlto-jobs=2 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; Test with all threads, on all cores, on all CPU sockets
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --thinlto-jobs=all %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --thinlto-jobs=all a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; Test with many more threads than the system has
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps --thinlto-jobs=100 %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps --thinlto-jobs=100 a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; Test with a bad value
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: not wasm-ld -r -save-temps --thinlto-jobs=foo %t1.o %t2.o -o %t3 2>&1 | FileCheck %s --check-prefix=BAD-JOBS
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: not wasm-ld -r -save-temps --thinlto-jobs=foo a.o d/b.o -o e/out 2>&1 | FileCheck %s --check-prefix=BAD-JOBS
; BAD-JOBS: error: --thinlto-jobs: invalid job count: foo
; Check without --thinlto-jobs (which currently defaults to heavyweight_hardware_concurrency, meanning one thread per hardware core -- not SMT)
-; RUN: rm -f %t31.lto.o %t32.lto.o
-; RUN: wasm-ld -r -save-temps %t1.o %t2.o -o %t3
-; RUN: llvm-nm %t31.lto.o | FileCheck %s --check-prefix=NM1
-; RUN: llvm-nm %t32.lto.o | FileCheck %s --check-prefix=NM2
+; RUN: rm -f out.lto.a.o d/out.lto.b.o
+; RUN: wasm-ld -r -save-temps a.o d/b.o -o e/out
+; RUN: llvm-nm out.lto.a.o | FileCheck %s --check-prefix=NM1
+; RUN: llvm-nm d/out.lto.b.o | FileCheck %s --check-prefix=NM2
; NM1: T f
; NM2: T g
diff --git a/lld/wasm/Config.h b/lld/wasm/Config.h
index 05a547ff9278a1..7be1927eb22421 100644
--- a/lld/wasm/Config.h
+++ b/lld/wasm/Config.h
@@ -79,6 +79,8 @@ struct Configuration {
// Because dyamanic linking under Wasm is still experimental we default to
// static linking
bool isStatic = true;
+ bool thinLTOEmitIndexFiles;
+ bool thinLTOIndexOnly;
bool trace;
uint64_t globalBase;
uint64_t initialHeap;
@@ -95,16 +97,18 @@ struct Configuration {
unsigned ltoo;
llvm::CodeGenOptLevel ltoCgo;
unsigned optimize;
- llvm::StringRef thinLTOJobs;
bool ltoDebugPassManager;
UnresolvedPolicy unresolvedSymbols;
BuildIdKind buildId = BuildIdKind::None;
llvm::StringRef entry;
+ llvm::StringRef ltoObjPath;
llvm::StringRef mapFile;
llvm::StringRef outputFile;
llvm::StringRef soName;
llvm::StringRef thinLTOCacheDir;
+ llvm::StringRef thinLTOJobs;
+ llvm::StringRef thinLTOIndexOnlyArg;
llvm::StringRef whyExtract;
llvm::StringSet<> allowUndefinedSymbols;
diff --git a/lld/wasm/Driver.cpp b/lld/wasm/Driver.cpp
index 9a27fc90457f08..9076a320a48e39 100644
--- a/lld/wasm/Driver.cpp
+++ b/lld/wasm/Driver.cpp
@@ -542,6 +542,7 @@ static void readConfigs(opt::InputArgList &args) {
else
error("invalid codegen optimization level for LTO: " + Twine(ltoCgo));
config->ltoPartitions = args::getInteger(args, OPT_lto_partitions, 1);
+ config->ltoObjPath = args.getLastArgValue(OPT_lto_obj_path_eq);
config->ltoDebugPassManager = args.hasArg(OPT_lto_debug_pass_manager);
config->mapFile = args.getLastArgValue(OPT_Map);
config->optimize = args::getInteger(args, OPT_O, 1);
@@ -569,6 +570,12 @@ static void readConfigs(opt::InputArgList &args) {
config->thinLTOCachePolicy = CHECK(
parseCachePruningPolicy(args.getLastArgValue(OPT_thinlto_cache_policy)),
"--thinlto-cache-policy: invalid cache policy");
+ config->thinLTOEmitIndexFiles = args.hasArg(OPT_thinlto_emit_index_files) ||
+ args.hasArg(OPT_thinlto_index_only) ||
+ args.hasArg(OPT_thinlto_index_only_eq);
+ config->thinLTOIndexOnly = args.hasArg(OPT_thinlto_index_only) ||
+ args.hasArg(OPT_thinlto_index_only_eq);
+ config->thinLTOIndexOnlyArg = args.getLastArgValue(OPT_thinlto_index_only_eq);
config->unresolvedSymbols = getUnresolvedSymbolPolicy(args);
config->whyExtract = args.getLastArgValue(OPT_why_extract);
errorHandler().verbose = args.hasArg(OPT_verbose);
@@ -1379,6 +1386,10 @@ void LinkerDriver::linkerMain(ArrayRef<const char *> argsArr) {
writeWhyExtract();
+ // Bail out if normal linked output is skipped due to LTO.
+ if (config->thinLTOIndexOnly)
+ return;
+
createOptionalSymbols();
// Resolve any variant symbols that were created due to signature
diff --git a/lld/wasm/LTO.cpp b/lld/wasm/LTO.cpp
index e523f0f6171535..c03483c2200689 100644
--- a/lld/wasm/LTO.cpp
+++ b/lld/wasm/LTO.cpp
@@ -11,7 +11,9 @@
#include "InputFiles.h"
#include "Symbols.h"
#include "lld/Common/Args.h"
+#include "lld/Common/CommonLinkerContext.h"
#include "lld/Common/ErrorHandler.h"
+#include "lld/Common/Filesystem.h"
#include "lld/Common/Strings.h"
#include "lld/Common/TargetOptionsCommandFlags.h"
#include "llvm/ADT/STLExtras.h"
@@ -27,6 +29,7 @@
#include "llvm/Support/Error.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
#include "llvm/Support/raw_ostream.h"
#include <algorithm>
#include <cstddef>
@@ -36,9 +39,10 @@
#include <vector>
using namespace llvm;
+using namespace lld::wasm;
+using namespace lld;
-namespace lld::wasm {
-static std::unique_ptr<lto::LTO> createLTO() {
+static lto::Config createConfig() {
lto::Config c;
c.Options = initTargetOptionsFromCodeGenFlags();
@@ -52,6 +56,7 @@ static std::unique_ptr<lto::LTO> createLTO() {
c.MAttrs = getMAttrs();
c.CGOptLevel = config->ltoCgo;
c.DebugPassManager = config->ltoDebugPassManager;
+ c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
if (config->relocatable)
c.RelocModel = std::nullopt;
@@ -63,13 +68,30 @@ static std::unique_ptr<lto::LTO> createLTO() {
if (config->saveTemps)
checkError(c.addSaveTemps(config->outputFile.str() + ".",
/*UseInputModulePath*/ true));
- lto::ThinBackend backend = lto::createInProcessThinBackend(
- llvm::heavyweight_hardware_concurrency(config->thinLTOJobs));
- return std::make_unique<lto::LTO>(std::move(c), backend,
- config->ltoPartitions);
+ return c;
}
-BitcodeCompiler::BitcodeCompiler() : ltoObj(createLTO()) {}
+namespace lld::wasm {
+
+BitcodeCompiler::BitcodeCompiler() {
+ // Initialize indexFile.
+ if (!config->thinLTOIndexOnlyArg.empty())
+ indexFile = openFile(config->thinLTOIndexOnlyArg);
+
+ // Initialize ltoObj.
+ lto::ThinBackend backend;
+ auto onIndexWrite = [&](StringRef s) { thinIndices.erase(s); };
+ if (config->thinLTOIndexOnly) {
+ backend = lto::createWriteIndexesThinBackend(
+ llvm::hardware_concurrency(config->thinLTOJobs),
+ "", "", "", false, indexFile.get(), onIndexWrite);
+ } else {
+ backend = lto::createInProcessThinBackend(
+ llvm::heavyweight_hardware_concurrency(config->thinLTOJobs));
+ }
+ ltoObj =
+ std::make_unique<lto::LTO>(createConfig(), backend, config->ltoPartitions);
+}
BitcodeCompiler::~BitcodeCompiler() = default;
@@ -90,6 +112,9 @@ void BitcodeCompiler::add(BitcodeFile &f) {
ArrayRef<Symbol *> syms = f.getSymbols();
std::vector<lto::SymbolResolution> resols(syms.size());
+ if (config->thinLTOEmitIndexFiles)
+ thinIndices.insert(obj.getName());
+
// Provide a resolution to the LTO API for each symbol.
for (const lto::InputFile::Symbol &objSym : obj.symbols()) {
Symbol *sym = syms[symNum];
@@ -136,25 +161,66 @@ std::vector<StringRef> BitcodeCompiler::compile() {
checkError(ltoObj->run(
[&](size_t task, const Twine &moduleName) {
+ buf[task].first = moduleName.str();
return std::make_unique<CachedFileStream>(
- std::make_unique<raw_svector_ostream>(buf[task]));
+ std::make_unique<raw_svector_ostream>(buf[task].second));
},
cache));
if (!config->thinLTOCacheDir.empty())
pruneCache(config->thinLTOCacheDir, config->thinLTOCachePolicy, files);
+ if (config->thinLTOIndexOnly) {
+ if (!config->ltoObjPath.empty())
+ saveBuffer(buf[0].second, config->ltoObjPath);
+
+ // ThinLTO with index only option is required to generate only the index
+ // files. After that, we exit from linker and ThinLTO backend runs in a
+ // distributed environment.
+ if (indexFile)
+ indexFile->close();
+ return {};
+ }
+
std::vector<StringRef> ret;
for (unsigned i = 0; i != maxTasks; ++i) {
- if (buf[i].empty())
+ StringRef objBuf = buf[i].second;
+ StringRef bitcodeFilePath = buf[i].first;
+ if (objBuf.empty())
+ continue;
+ ret.emplace_back(objBuf.data(), objBuf.size());
+ if (!config->saveTemps)
continue;
- if (config->saveTemps) {
- if (i == 0)
- saveBuffer(buf[i], config->outputFile + ".lto.o");
- else
- saveBuffer(buf[i], config->outputFile + Twine(i) + ".lto.o");
+
+ // If the input bitcode file is path/to/x.o and -o specifies a.out, the
+ // corresponding native relocatable file path will look like:
+ // path/to/a.out.lto.x.o.
+ StringRef ltoObjName;
+ if (bitcodeFilePath == "ld-temp.o") {
+ ltoObjName =
+ saver().save(Twine(config->outputFile) + ".lto" +
+ (i == 0 ? Twine("") : Twine('.') + Twine(i)) + ".o");
+ } else {
+ StringRef directory = sys::path::parent_path(bitcodeFilePath);
+ // For an archive member, which has an identifier like "d/a.a(coll.o at
+ // 8)" (see BitcodeFile::BitcodeFile), use the filename; otherwise, use
+ // the stem (d/a.o => a).
+ StringRef baseName = bitcodeFilePath.ends_with(")")
+ ? sys::path::filename(bitcodeFilePath) : sys::path::stem(bitcodeFilePath);
+ StringRef outputFileBaseName = sys::path::filename(config->outputFile);
+ SmallString<256> path;
+ sys::path::append(path, directory,
+ outputFileBaseName + ".lto." + baseName + ".o");
+ sys::path::remove_dots(path, true);
+ ltoObjName = saver().save(path.str());
}
- ret.emplace_back(buf[i].data(), buf[i].size());
+ saveBuffer(objBuf, ltoObjName);
+ }
+
+ if (!config->ltoObjPath.empty()) {
+ saveBuffer(buf[0].second, config->ltoObjPath);
+ for (unsigned i = 1; i != maxTasks; ++i)
+ saveBuffer(buf[i].second, config->ltoObjPath + Twine(i));
}
for (std::unique_ptr<MemoryBuffer> &file : files)
diff --git a/lld/wasm/LTO.h b/lld/wasm/LTO.h
index bb57c6651394b7..c61faf82b8fe6d 100644
--- a/lld/wasm/LTO.h
+++ b/lld/wasm/LTO.h
@@ -21,7 +21,9 @@
#define LLD_WASM_LTO_H
#include "lld/Common/LLVM.h"
+#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/raw_ostream.h"
#include "Writer.h"
#include <memory>
#include <vector>
@@ -47,8 +49,11 @@ class BitcodeCompiler {
private:
std::unique_ptr<llvm::lto::LTO> ltoObj;
- std::vector<SmallString<0>> buf;
+ // An array of (module name, native relocatable file content) pairs.
+ SmallVector<std::pair<std::string, SmallString<0>>, 0> buf;
std::vector<std::unique_ptr<MemoryBuffer>> files;
+ std::unique_ptr<llvm::raw_fd_ostream> indexFile;
+ llvm::DenseSet<StringRef> thinIndices;
};
} // namespace lld::wasm
diff --git a/lld/wasm/Options.td b/lld/wasm/Options.td
index cff29e709a1a09..2ffe79f970f1fe 100644
--- a/lld/wasm/Options.td
+++ b/lld/wasm/Options.td
@@ -297,11 +297,15 @@ def lto_CGO: JJ<"lto-CGO">, MetaVarName<"<cgopt-level>">,
HelpText<"Codegen optimization level for LTO">;
def lto_partitions: JJ<"lto-partitions=">,
HelpText<"Number of LTO codegen partitions">;
+def lto_obj_path_eq: JJ<"lto-obj-path=">;
def disable_verify: F<"disable-verify">;
def save_temps: F<"save-temps">, HelpText<"Save intermediate LTO compilation results">;
def thinlto_cache_dir: JJ<"thinlto-cache-dir=">,
HelpText<"Path to ThinLTO cached object file directory">;
defm thinlto_cache_policy: EEq<"thinlto-cache-policy", "Pruning policy for the ThinLTO cache">;
+def thinlto_emit_index_files: FF<"thinlto-emit-index-files">;
+def thinlto_index_only: FF<"thinlto-index-only">;
+def thinlto_index_only_eq: JJ<"thinlto-index-only=">;
def thinlto_jobs: JJ<"thinlto-jobs=">,
HelpText<"Number of ThinLTO jobs. Default to --threads=">;
def lto_debug_pass_manager: FF<"lto-debug-pass-manager">,
|
sbc100
changed the title
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
dschuff
approved these changes
Nov 1, 2024
The changes in this PR (both in the code and the tests) are largely copied directly from the ELF linker. Parial fix for llvm#79604.
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/59/builds/7487 Here is the relevant piece of the build log for the reference |
smallp-o-p
pushed a commit
to smallp-o-p/llvm-project
that referenced
this pull request
Nov 3, 2024
The changes in this PR (both in the code and the tests) are largely copied directly from the ELF linker. Partial fix for llvm#79604.
NoumanAmir657
pushed a commit
to NoumanAmir657/llvm-project
that referenced
this pull request
Nov 4, 2024
The changes in this PR (both in the code and the tests) are largely copied directly from the ELF linker. Partial fix for llvm#79604.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The changes in this PR (both in the code and the tests) are largely copied directly from the ELF linker.
Partial fix for #79604.