Skip to content

Move a lot of symbol code to use the symbol string pool #115796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
ef47492
[jit] Moving all the symbol instances to be backed by a single pool
jaredwy Nov 28, 2024
218bad7
Update JITLinkLinker.cpp
lhames Dec 4, 2024
02e154c
Fix typo.
lhames Dec 4, 2024
a983fc9
Remove deref.
lhames Dec 4, 2024
1dd3409
Remove SymbolStringPtr derefs in debugging output.
lhames Dec 4, 2024
10e98cb
Rename imageBaseName -> ImageBaseName, make method const.
lhames Dec 4, 2024
7e51800
Remove SymbolStringPtr derefs in debugging output.
lhames Dec 4, 2024
4c066ab
Remove SymbolStringPtr derefs in debugging output.
lhames Dec 4, 2024
0233614
Remove SymbolStringPtr deref operation.
lhames Dec 4, 2024
5829c65
Merge branch 'main' into wyles/symbolpool-switch
lhames Dec 4, 2024
1b37fef
Remove SymbolStringPtr deref in debugging output.
lhames Dec 4, 2024
855582c
Remove redundant formatv.
lhames Dec 4, 2024
ce642a6
Remove SymbolStringPtr deref in debugging output.
lhames Dec 4, 2024
c2f8cd6
Remove SymbolStringPtr deref in debugging output.
lhames Dec 4, 2024
473c925
Remove SymbolStringPtr deref in debugging output.
lhames Dec 4, 2024
0de822d
Remove SymbolStringPtr deref operation in debugging output.
lhames Dec 4, 2024
a71d319
Remove SymbolStringPtr deref in debugging output.
lhames Dec 4, 2024
c9c9286
Remove SymbolStringPtr derefs in debugging output.
lhames Dec 4, 2024
a52db68
Remove SymbolStringPtr deref in debugging output.
lhames Dec 4, 2024
2383294
[jit] Use the symbolpool from the es in the memory manager
jaredwy Dec 4, 2024
ab63e6a
Git Format changes
jaredwy Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ autoconf/autom4te.cache
# VS2017 and VSCode config files.
.vscode
.vs
#zed config files
.zed
Comment on lines +62 to +63
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftover change? (If this is useful we should commit it separately)

# pythonenv for github Codespaces
pythonenv*
# clangd index. (".clangd" is a config file now, thus trailing slash)
Expand Down
19 changes: 12 additions & 7 deletions bolt/include/bolt/Core/BinaryContext.h
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
#include "llvm/ADT/iterator.h"
#include "llvm/BinaryFormat/Dwarf.h"
#include "llvm/BinaryFormat/MachO.h"
#include "llvm/ExecutionEngine/Orc/SymbolStringPool.h"
#include "llvm/MC/MCAsmInfo.h"
#include "llvm/MC/MCCodeEmitter.h"
#include "llvm/MC/MCContext.h"
Expand Down Expand Up @@ -276,11 +277,10 @@ class BinaryContext {
void deregisterSectionName(const BinarySection &Section);

public:
static Expected<std::unique_ptr<BinaryContext>>
createBinaryContext(Triple TheTriple, StringRef InputFileName,
SubtargetFeatures *Features, bool IsPIC,
std::unique_ptr<DWARFContext> DwCtx,
JournalingStreams Logger);
static Expected<std::unique_ptr<BinaryContext>> createBinaryContext(
Triple TheTriple, std::shared_ptr<orc::SymbolStringPool> SSP,
StringRef InputFileName, SubtargetFeatures *Features, bool IsPIC,
std::unique_ptr<DWARFContext> DwCtx, JournalingStreams Logger);

/// Superset of compiler units that will contain overwritten code that needs
/// new debug info. In a few cases, functions may end up not being
Expand Down Expand Up @@ -372,6 +372,7 @@ class BinaryContext {
bool hasSymbolsWithFileName() const { return HasSymbolsWithFileName; }
void setHasSymbolsWithFileName(bool Value) { HasSymbolsWithFileName = Value; }

std::shared_ptr<orc::SymbolStringPool> getSymbolStringPool() { return SSP; }
/// Return true if relocations against symbol with a given name
/// must be created.
bool forceSymbolRelocations(StringRef SymbolName) const;
Expand Down Expand Up @@ -631,6 +632,8 @@ class BinaryContext {

std::unique_ptr<Triple> TheTriple;

std::shared_ptr<orc::SymbolStringPool> SSP;

const Target *TheTarget;

std::string TripleName;
Expand Down Expand Up @@ -807,8 +810,10 @@ class BinaryContext {

BinaryContext(std::unique_ptr<MCContext> Ctx,
std::unique_ptr<DWARFContext> DwCtx,
std::unique_ptr<Triple> TheTriple, const Target *TheTarget,
std::string TripleName, std::unique_ptr<MCCodeEmitter> MCE,
std::unique_ptr<Triple> TheTriple,
std::shared_ptr<orc::SymbolStringPool> SSP,
const Target *TheTarget, std::string TripleName,
std::unique_ptr<MCCodeEmitter> MCE,
std::unique_ptr<MCObjectFileInfo> MOFI,
std::unique_ptr<const MCAsmInfo> AsmInfo,
std::unique_ptr<const MCInstrInfo> MII,
Expand Down
10 changes: 6 additions & 4 deletions bolt/lib/Core/BinaryContext.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ void BinaryContext::logBOLTErrorsAndQuitOnFatal(Error E) {
BinaryContext::BinaryContext(std::unique_ptr<MCContext> Ctx,
std::unique_ptr<DWARFContext> DwCtx,
std::unique_ptr<Triple> TheTriple,
std::shared_ptr<orc::SymbolStringPool> SSP,
const Target *TheTarget, std::string TripleName,
std::unique_ptr<MCCodeEmitter> MCE,
std::unique_ptr<MCObjectFileInfo> MOFI,
Expand All @@ -136,7 +137,7 @@ BinaryContext::BinaryContext(std::unique_ptr<MCContext> Ctx,
std::unique_ptr<MCDisassembler> DisAsm,
JournalingStreams Logger)
: Ctx(std::move(Ctx)), DwCtx(std::move(DwCtx)),
TheTriple(std::move(TheTriple)), TheTarget(TheTarget),
TheTriple(std::move(TheTriple)), SSP(std::move(SSP)), TheTarget(TheTarget),
TripleName(TripleName), MCE(std::move(MCE)), MOFI(std::move(MOFI)),
AsmInfo(std::move(AsmInfo)), MII(std::move(MII)), STI(std::move(STI)),
InstPrinter(std::move(InstPrinter)), MIA(std::move(MIA)),
Expand All @@ -159,8 +160,9 @@ BinaryContext::~BinaryContext() {
/// Create BinaryContext for a given architecture \p ArchName and
/// triple \p TripleName.
Expected<std::unique_ptr<BinaryContext>> BinaryContext::createBinaryContext(
Triple TheTriple, StringRef InputFileName, SubtargetFeatures *Features,
bool IsPIC, std::unique_ptr<DWARFContext> DwCtx, JournalingStreams Logger) {
Triple TheTriple, std::shared_ptr<orc::SymbolStringPool> SSP,
StringRef InputFileName, SubtargetFeatures *Features, bool IsPIC,
std::unique_ptr<DWARFContext> DwCtx, JournalingStreams Logger) {
StringRef ArchName = "";
std::string FeaturesStr = "";
switch (TheTriple.getArch()) {
Expand Down Expand Up @@ -283,7 +285,7 @@ Expected<std::unique_ptr<BinaryContext>> BinaryContext::createBinaryContext(

auto BC = std::make_unique<BinaryContext>(
std::move(Ctx), std::move(DwCtx), std::make_unique<Triple>(TheTriple),
TheTarget, std::string(TripleName), std::move(MCE), std::move(MOFI),
std::move(SSP), TheTarget, std::string(TripleName), std::move(MCE), std::move(MOFI),
std::move(AsmInfo), std::move(MII), std::move(STI),
std::move(InstructionPrinter), std::move(MIA), nullptr, std::move(MRI),
std::move(DisAsm), Logger);
Expand Down
3 changes: 2 additions & 1 deletion bolt/lib/Rewrite/DWARFRewriter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1691,7 +1691,8 @@ namespace {
std::unique_ptr<BinaryContext>
createDwarfOnlyBC(const object::ObjectFile &File) {
return cantFail(BinaryContext::createBinaryContext(
File.makeTriple(), File.getFileName(), nullptr, false,
File.makeTriple(), std::make_shared<orc::SymbolStringPool>(),
File.getFileName(), nullptr, false,
DWARFContext::create(File, DWARFContext::ProcessDebugRelocations::Ignore,
nullptr, "", WithColor::defaultErrorHandler,
WithColor::defaultWarningHandler),
Expand Down
10 changes: 7 additions & 3 deletions bolt/lib/Rewrite/JITLinkLinker.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ struct JITLinkLinker::Context : jitlink::JITLinkContext {
jitlink::AsyncLookupResult AllResults;

for (const auto &Symbol : Symbols) {
std::string SymName = Symbol.first.str();
std::string SymName = (*Symbol.first).str();
LLVM_DEBUG(dbgs() << "BOLT: looking for " << SymName << "\n");

if (auto Address = Linker.lookupSymbol(SymName)) {
Expand Down Expand Up @@ -167,7 +167,11 @@ struct JITLinkLinker::Context : jitlink::JITLinkContext {
Error notifyResolved(jitlink::LinkGraph &G) override {
for (auto *Symbol : G.defined_symbols()) {
SymbolInfo Info{Symbol->getAddress().getValue(), Symbol->getSize()};
Linker.Symtab.insert({Symbol->getName().str(), Info});
auto Name = Symbol->getName();
std::string NameStr("");
if (Name)
NameStr = (*Name).str();
Linker.Symtab.insert({NameStr, Info});
}

return Error::success();
Expand All @@ -189,7 +193,7 @@ JITLinkLinker::~JITLinkLinker() { cantFail(MM->deallocate(std::move(Allocs))); }

void JITLinkLinker::loadObject(MemoryBufferRef Obj,
SectionsMapper MapSections) {
auto LG = jitlink::createLinkGraphFromObject(Obj);
auto LG = jitlink::createLinkGraphFromObject(Obj, BC.getSymbolStringPool());
if (auto E = LG.takeError()) {
errs() << "BOLT-ERROR: JITLink failed: " << E << '\n';
exit(1);
Expand Down
3 changes: 2 additions & 1 deletion bolt/lib/Rewrite/MachORewriteInstance.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,8 @@ MachORewriteInstance::MachORewriteInstance(object::MachOObjectFile *InputFile,
ErrorAsOutParameter EAO(&Err);
Relocation::Arch = InputFile->makeTriple().getArch();
auto BCOrErr = BinaryContext::createBinaryContext(
InputFile->makeTriple(), InputFile->getFileName(), nullptr,
InputFile->makeTriple(), std::make_shared<orc::SymbolStringPool>(),
InputFile->getFileName(), nullptr,
/* IsPIC */ true, DWARFContext::create(*InputFile),
{llvm::outs(), llvm::errs()});
if (Error E = BCOrErr.takeError()) {
Expand Down
3 changes: 2 additions & 1 deletion bolt/lib/Rewrite/RewriteInstance.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -356,7 +356,8 @@ RewriteInstance::RewriteInstance(ELFObjectFileBase *File, const int Argc,

Relocation::Arch = TheTriple.getArch();
auto BCOrErr = BinaryContext::createBinaryContext(
TheTriple, File->getFileName(), Features.get(), IsPIC,
TheTriple, std::make_shared<orc::SymbolStringPool>(),
File->getFileName(), Features.get(), IsPIC,
DWARFContext::create(*File, DWARFContext::ProcessDebugRelocations::Ignore,
nullptr, opts::DWPPathName,
WithColor::defaultErrorHandler,
Expand Down
5 changes: 3 additions & 2 deletions bolt/unittests/Core/BinaryContext.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ struct BinaryContextTester : public testing::TestWithParam<Triple::ArchType> {
void initializeBOLT() {
Relocation::Arch = ObjFile->makeTriple().getArch();
BC = cantFail(BinaryContext::createBinaryContext(
ObjFile->makeTriple(), ObjFile->getFileName(), nullptr, true,
ObjFile->makeTriple(), std::make_shared<orc::SymbolStringPool>(),
ObjFile->getFileName(), nullptr, true,
DWARFContext::create(*ObjFile.get()), {llvm::outs(), llvm::errs()}));
ASSERT_FALSE(!BC);
}
Expand Down Expand Up @@ -216,4 +217,4 @@ TEST_P(BinaryContextTester, BaseAddressSegmentsSmallerThanAlignment) {
BC->getBaseAddressForMapping(0xaaaaaaab1000, 0x1000);
ASSERT_TRUE(BaseAddress.has_value());
ASSERT_EQ(*BaseAddress, 0xaaaaaaaa0000ULL);
}
}
3 changes: 2 additions & 1 deletion bolt/unittests/Core/MCPlusBuilder.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ struct MCPlusBuilderTester : public testing::TestWithParam<Triple::ArchType> {
void initializeBolt() {
Relocation::Arch = ObjFile->makeTriple().getArch();
BC = cantFail(BinaryContext::createBinaryContext(
ObjFile->makeTriple(), ObjFile->getFileName(), nullptr, true,
ObjFile->makeTriple(), std::make_shared<orc::SymbolStringPool>(),
ObjFile->getFileName(), nullptr, true,
DWARFContext::create(*ObjFile.get()), {llvm::outs(), llvm::errs()}));
ASSERT_FALSE(!BC);
BC->initializeTarget(std::unique_ptr<MCPlusBuilder>(
Expand Down
3 changes: 2 additions & 1 deletion llvm/include/llvm/ExecutionEngine/JITLink/COFF.h
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ namespace jitlink {
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromCOFFObject(MemoryBufferRef ObjectBuffer);
createLinkGraphFromCOFFObject(MemoryBufferRef ObjectBuffer,
std::shared_ptr<orc::SymbolStringPool> SSP);

/// Link the given graph.
///
Expand Down
4 changes: 2 additions & 2 deletions llvm/include/llvm/ExecutionEngine/JITLink/COFF_x86_64.h
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ namespace jitlink {
/// Note: The graph does not take ownership of the underlying buffer, nor copy
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromCOFFObject_x86_64(MemoryBufferRef ObjectBuffer);
Expected<std::unique_ptr<LinkGraph>> createLinkGraphFromCOFFObject_x86_64(
MemoryBufferRef ObjectBuffer, std::shared_ptr<orc::SymbolStringPool> SSP);

/// jit-link the given object buffer, which must be a COFF x86-64 object file.
void link_COFF_x86_64(std::unique_ptr<LinkGraph> G,
Expand Down
3 changes: 2 additions & 1 deletion llvm/include/llvm/ExecutionEngine/JITLink/ELF.h
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ namespace jitlink {
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject(MemoryBufferRef ObjectBuffer);
createLinkGraphFromELFObject(MemoryBufferRef ObjectBuffer,
std::shared_ptr<orc::SymbolStringPool> SSP);

/// Link the given graph.
///
Expand Down
4 changes: 2 additions & 2 deletions llvm/include/llvm/ExecutionEngine/JITLink/ELF_aarch32.h
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ namespace jitlink {
/// Note: The graph does not take ownership of the underlying buffer, nor copy
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject_aarch32(MemoryBufferRef ObjectBuffer);
Expected<std::unique_ptr<LinkGraph>> createLinkGraphFromELFObject_aarch32(
MemoryBufferRef ObjectBuffer, std::shared_ptr<orc::SymbolStringPool> SSP);

/// jit-link the given object buffer, which must be an ELF arm/thumb object
/// file.
Expand Down
4 changes: 2 additions & 2 deletions llvm/include/llvm/ExecutionEngine/JITLink/ELF_aarch64.h
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ namespace jitlink {
/// Note: The graph does not take ownership of the underlying buffer, nor copy
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject_aarch64(MemoryBufferRef ObjectBuffer);
Expected<std::unique_ptr<LinkGraph>> createLinkGraphFromELFObject_aarch64(
MemoryBufferRef ObjectBuffer, std::shared_ptr<orc::SymbolStringPool> SSP);

/// jit-link the given object buffer, which must be a ELF aarch64 relocatable
/// object file.
Expand Down
3 changes: 2 additions & 1 deletion llvm/include/llvm/ExecutionEngine/JITLink/ELF_i386.h
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ namespace jitlink {
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject_i386(MemoryBufferRef ObjectBuffer);
createLinkGraphFromELFObject_i386(MemoryBufferRef ObjectBuffer,
std::shared_ptr<orc::SymbolStringPool> SSP);

/// jit-link the given object buffer, which must be a ELF i386 relocatable
/// object file.
Expand Down
4 changes: 2 additions & 2 deletions llvm/include/llvm/ExecutionEngine/JITLink/ELF_loongarch.h
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ namespace jitlink {
/// Note: The graph does not take ownership of the underlying buffer, nor copy
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject_loongarch(MemoryBufferRef ObjectBuffer);
Expected<std::unique_ptr<LinkGraph>> createLinkGraphFromELFObject_loongarch(
MemoryBufferRef ObjectBuffer, std::shared_ptr<orc::SymbolStringPool> SSP);

/// jit-link the given object buffer, which must be an ELF loongarch object
/// file.
Expand Down
7 changes: 4 additions & 3 deletions llvm/include/llvm/ExecutionEngine/JITLink/ELF_ppc64.h
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,16 @@ namespace llvm::jitlink {
///
/// WARNING: The big-endian backend has not been tested yet.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject_ppc64(MemoryBufferRef ObjectBuffer);
createLinkGraphFromELFObject_ppc64(MemoryBufferRef ObjectBuffer,
std::shared_ptr<orc::SymbolStringPool> SSP);

/// Create a LinkGraph from an ELF/ppc64le relocatable object.
///
/// Note: The graph does not take ownership of the underlying buffer, nor copy
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject_ppc64le(MemoryBufferRef ObjectBuffer);
Expected<std::unique_ptr<LinkGraph>> createLinkGraphFromELFObject_ppc64le(
MemoryBufferRef ObjectBuffer, std::shared_ptr<orc::SymbolStringPool> SSP);

/// jit-link the given object buffer, which must be a ELF ppc64le object file.
///
Expand Down
3 changes: 2 additions & 1 deletion llvm/include/llvm/ExecutionEngine/JITLink/ELF_riscv.h
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ namespace jitlink {
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject_riscv(MemoryBufferRef ObjectBuffer);
createLinkGraphFromELFObject_riscv(MemoryBufferRef ObjectBuffer,
std::shared_ptr<orc::SymbolStringPool> SSP);

/// jit-link the given object buffer, which must be a ELF riscv object file.
void link_ELF_riscv(std::unique_ptr<LinkGraph> G,
Expand Down
3 changes: 2 additions & 1 deletion llvm/include/llvm/ExecutionEngine/JITLink/ELF_x86_64.h
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ namespace jitlink {
/// its contents. The caller is responsible for ensuring that the object buffer
/// outlives the graph.
Expected<std::unique_ptr<LinkGraph>>
createLinkGraphFromELFObject_x86_64(MemoryBufferRef ObjectBuffer);
createLinkGraphFromELFObject_x86_64(MemoryBufferRef ObjectBuffe,
std::shared_ptr<orc::SymbolStringPool> SSP);

/// jit-link the given object buffer, which must be a ELF x86-64 object file.
void link_ELF_x86_64(std::unique_ptr<LinkGraph> G,
Expand Down
Loading
Loading