Talk:Position-independent code

Unnamed section

Note that although "position-independent code" is arguably more grammatical, common usage strongly favor the no-hyphen version, presumably because system programmers are only semi-literate. :-) Stan 14:15, 2 Apr 2004 (UTC)

Cesian programs

I noticed that Evan added a link to to question the following sentence: However, [PIC] can be generated automatically with a Cesian program. The text was added by User:Mirad, who has only a few contributions. I'm going to go out on a limb and say I think the term is bogus. -- Wmahan. 22:09, 13 October 2005 (UTC)

ELF

Though this article doesn't mention it, nearly all of the details are basically specific to ELF, aren't they? I think this should be made clearer.

MikeHearn 21:27, 14 May 2006 (UTC)

The ELF mechanisms are based on those used in SunOS 4.x, which used an a.out-based format. I think Mach-O in Mac OS X uses mechanisms that are similar in some ways, and perhaps the mechanisms used with 32-bit PA-RISC in HP-UX and used in Tru64 UNIX, and even the mechanisms used in AIX, are also similar, but, yes, the page should probably discuss that, perhaps giving terminological and technical differences between various UN*Xes. Guy Harris 22:06, 14 May 2006 (UTC)

I've renamed the "Technical details" section to "SunOS 4.x and ELF", and added a reference to the original paper about the SunOS 4.x implementation. Guy Harris (talk) 21:03, 23 June 2024 (UTC)

Relocation Work Required

Although relocatable code requires much more fixup than does PIC, PIC still requires some fixup by the loader. For example when using a GOT, the GOT can contain absolute addresses that need to be fixed up based on where the module is loaded.

Yes, SunOS 4-style PIC (which is what ELF PIC is, and Mach-O PIC is probably influenced by SunOS 4 as well, given that the person who did a lot of work on the SunOS 4 dynamic linking went to NeXTno, the person I was thinking of wasn't mentioned in the Gingell et al paper on SunOS 4.x shared libraries, and NeXTStEP may have had a non-PIC shared library mechanism) add the GOT and PLT to point to of external symbols, so that the code that uses them doesn't need to be modified to relocate it and can be shared. Guy Harris (talk) 20:10, 23 June 2024 (UTC)

What functions are inherited?

As this page is linked from Library (computing) it should be somewhat more explicative.

In particular, the following sentence leaves much to be guessed:

This notably allows a shared library to inherit certain function calls from previously loaded libraries rather than e. g. using its own versions

Why is that an advantage? E.g., Can an executable that exports its own malloc() propagate it to loaded libraries? On the other hand, why can that inheritance be a problem? It may be useful to include a link to best programming practices or gotchas one should watch against. This is an example where Unix and Windows differ. Those differences are mentioned often, but is difficult to understand their semantic implications.

How does such inheritance propagate across the exec family of functions? In general, an executable can behave differently depending on which process loaded it, because the parent process can have a number of shared libraries already loaded in its address space. Where can one learn more about this topic?

194.243.254.190 11:44, 3 December 2006 (UTC)

History: TSS

I'd like to say something about the TSS/360 implementation of PIC which, to me, sounds a lot like current Linux implementations, but I'm not sure I have enough background. What earlier implementations existed? Peter Flass (talk) 11:43, 17 October 2013 (UTC)

PIC and operating systems

In "History" I eliminated the paragraph concerning the use of PIC to implement operating system overlays. For example OS/360 used SVC transient areas, but not necessarily PIC (type 1 and 2 SVCs). The Burroughs 5500 MCP falls into the category of segmented systems and was inherently PIC. Peter Flass (talk) 12:20, 17 October 2013 (UTC)

I haven't thought about SVC transient routines for some years. As well as I remember, they are less than 4K, so should be able to address relative to the entry point. I believe also that they are reentrant, and so can't store into their space. Gah4 (talk) 05:57, 21 June 2024 (UTC)

In OS/360 SVC transient areas are 1 KiB; in OS/VS1 they are 2 KiB and in OS/VS2 transient SVCs live in the LPA and there are no SVC transient areas. In all three cases SVC transient routines are supposed to be refreshable and reentrant.

The OS preloads several base registers for SVCs; note that the OS does not prload a register with the address of the SVC routine.

In practice, a transient SVC routine can get by with a single base register for the code. -- Shmuel (Seymour J.) Metz Username:Chatul (talk) 09:46, 21 June 2024 (UTC)

Many years ago, I had the Fixed Task Supervisor PLM which is probably the most readable of manuals describing the internals of OS/360. That is, no complications of MFT or MVT. Gah4 (talk) 21:13, 21 June 2024 (UTC)

I found the MVT logic manuals to be quite good, although I need to refer to the PCP manuals for some details. -- Shmuel (Seymour J.) Metz Username:Chatul (talk) 05:19, 23 June 2024 (UTC)

Notes

References

Windows

I removed some incorrect and irrelevant information about Windows and the relationship between PIC code and pre-linking. Pre-linking doesn't have much to do with PIC code. It has more to do with symbol resolution, which is for references that could potentially cross modules. PIC is more about references within a module.— Preceding unsigned comment added by 2001:4898:80e0:ee43::3 (talk) 00:06, 31 January 2015 (UTC)‎

Link only first use or also first in section?

Position-independent code#History has links for Multics and TSS/360. A recent edit by 70.92.191.178 added a second link for Multics, in the Multics section; there is no second link for TSS. Should I also link TSS/360 in the TSS section, or revert the edit? Shmuel (Seymour J.) Metz Username:Chatul (talk) 18:19, 6 July 2020 (UTC)

WP:NPOV

The section #Technical details describes a specific implementation, and as such should either be under the platform to which it applies or include the platform in the section name. In particular, not all implementations of PIC use a global offset table (GOT). --Shmuel (Seymour J.) Metz Username:Chatul (talk) 17:17, 6 September 2021 (UTC)

It's more of an over-specificity issue than a neutrality issue; it's not saying this is the only right way to do PIC, it's saying "this is the way PIC is done" rather than, more specifically, "this is the way PIC is done in systems implemented using the SunOS 4.0 style of PIC and dynamic linking" (SVR4's system was based on the SunOS 4.0 one, and other systems such as Linux and the *BSDs that adopted SVR4's ELF also adopted its dynamic linking system; some other UN*Xes also modeled their systems after the SunOS 4.0 system). Guy Harris (talk) 19:04, 6 September 2021 (UTC)

I think the endless disputes about how PIC works and what OSes are inferior and so on stem from conflating two terms/ideas under one command-line option (-fpic) in GCC: position independent code and interposition of symbols in ELF format. The later is used to make sure if a symbol is defined in multiple binaries then the earliest definition "wins" and a binary will use the winning definition even though the binary already contains its own implementation. Sadly the second concept has no page on the Wikipedia. And I do not think it is Wikipedia's job to untangle the misunderstanding. --Alvin-cs ✉ 17:53, 15 October 2021 (UTC)

PC relative addressing

As part of another article, it seems it would be useful to have an article on PC relative addressing. That makes PIC easier, and also simplifies some other addressing problems. It could go in Addressing mode, but I don't think that is the best place. It could go here, or have its own page. Gah4 (talk) 00:41, 21 June 2024 (UTC)

There's already Addressing mode § PC-relative and Addressing mode § PC-relative 2. Is that not sufficient? Guy Harris (talk) 21:43, 21 June 2024 (UTC)

The reason for the comment, is that I believe it is important enough to have its own article. One advantage, as noted, is the ease of writing PIC. (That is, for PC relative branching.) As I understand it, many processors can't put the same cache block into the data and code cache at the same time. That could be mentioned along with discussion of PC relative data addressing. I believe for ESA/390 processors, you are supposed to keep at least 256 bytes between instructions and data. Gah4 (talk) 00:58, 22 June 2024 (UTC)

I believe it is important enough to have its own article Why?

One advantage, as noted, is the ease of writing PIC. (That is, for PC relative branching.) As I understand it, many processors can't put the same cache block into the data and code cache at the same time. But why does that justify a page for a particular type of addressing mode (which already appears in addressing mode), rather than, say, a note on position-independent code?

As I understand it, many processors can't put the same cache block into the data and code cache at the same time. [citation needed]. If the processor either flushes or updates the I-cache if a store is done into a location in the I-cache (I think that's the case for x86, for backward compatibility, and I suspect it's true for any split-I&D-cache {S/3x0, z/Architecture} processor, for the same reason), I don't see why the same cache line couldn't be in both caches. If the processor requires an explicit I-cache flush (which at least one SPARC v8 processor does), I'm not sure I see a problem with a store into a data location as long as it's not later used as an instruction, but in that case you'd need to do an I-cache flush anyway.

That could be mentioned along with discussion of PC relative data addressing. PC-relative data addressing is used heavily in UN*Xes; references to external data in PIC are done as indirect references through entries in the Global Offset Table (GOT), and the GOT for a module is at a fixed offset from the code for that module, known at the time the module is built, so references to the GOT are done with PC-relative (or, if PC relative data addressing isn't supported, beginning-of-the-routine-relative) addressing. The GOT and code aren't intermixed - there's a big blob of code and a smaller blob of GOT entries containing addresses of globals - so you won't see code and a GOT entry in the same cache line (or the same page. The data section of a module is also at a known-at-build-time offset from the code section, so PC-relative (or beginning-of-the-routine-relative) addressing can be used to refer to that data. Guy Harris (talk) 07:06, 22 June 2024 (UTC)

Reminds me of what I used to do every time a new release of SunOS came out. As designed, SunOS uses YP/NIS for name resolution, not DNS. Sun supplies a PIC version of libc and the resolver routines, which ones has to link together to generate a new libc.so. But I never looked in detail of what was in the PIC version, especially as they didn't supply source. Gah4 (talk) 11:21, 22 June 2024 (UTC)

What is PIC?

On a system such as Multics or TSS/360, a shared program may appear at different virtual addresses in different processes. Similarly, there are systems without virtual memory where code may be moved transparently. The term PIC is clearly appropriate for these programs.

However, for some systems with virtual memory, programs contain address constants that refer to fixed (after loading) virtual addresses, and I know of no sources that refer to them as PIC. -- Shmuel (Seymour J.) Metz Username:Chatul (talk) 14:00, 24 June 2024 (UTC)

a shared program may appear at different virtual addresses in different processes Does "program" include shared libraries? If so, they can appear at different virtual addresses in different processes in most UN*Xes that support them.

Similarly, on UN*Xes that support position-independent executables, a position-independent executable usually appears at different virtual addresses in different processes, to foil attacks that assume the program will be loaded at the same virtual address whenever it's run (address space layout randomization).

for some systems with virtual memory, programs contain address constants that refer to fixed (after loading) virtual addresses Which systems are you thinking of here? Guy Harris (talk) 23:25, 18 July 2024 (UTC)

In Multics and TSS/360, there is no need for a DLL; everything is dynamic. Every program is in two pieces, one of which contains read-only code and no address constants. The other piece is called a linkage segment in Multics and is a prototype section (PSECT) in TSS. Only the piece containing the code is shared. Both systems have very different memory and process models from *ix systems.

In OS/VS1 and OS/VS2, the programs in the Link Pack Area (LPA) are shared and are not position independent. Programs dynamically loaded in a partition or region are relocated on the initial load and are then remain at the same virtual address for every task within the partition or region. As with Multics and TSS, the memory and process models are very different from *ix.-- Shmuel (Seymour J.) Metz Username:Chatul (talk) 03:10, 19 July 2024 (UTC)

Every program is in two pieces, one of which contains read-only code and no address constants. To quote what the article says about SunOS 4.x position-independent code and position-independent code on systems using ELF as an executable format:

Procedure calls inside a shared library are typically made through small procedure linkage table (PLT) stubs, which then call the definitive function. This notably allows a shared library to inherit certain function calls from previously loaded libraries rather than using its own versions.

Data references from position-independent code are usually made indirectly, through Global Offset Tables (GOTs), which store the addresses of all accessed global variables. There is one GOT per compilation unit or object module, and it is located at a fixed offset from the code (although this offset is not known until the library is linked). When a linker links modules to create a shared library, it merges the GOTs and sets the final offsets in code. It is not necessary to adjust the offsets when loading the shared library later.

so executable code in both dynamic shared libraries and position-independent executables has read-only code in the text section, patchable code for procedure calls outside the library/file in the procedure linkage table PLT (PLT), and what would be considered address constants in the global offset table (GOT). If I remember correctly, both the PLT and the GOT are loaded at an offset, calculated at link time (i.e., at the time one or more object files are linked into a shared library or executable), from where the code segment is loaded, and the code refers to the PLT (in procedure calls through the PLT entry) and GOT (in references to global variables) using either PC-relative or "the address of an instruction in the function prolog that puts its address or the address of the next instruction into a register, which is used as an index register, so it's relative to that instruction's address" addressing. This means that all addressing in the code section is position-independent and needs no patching at startup time to change addresses.

The other piece is called a linkage segment in Multics and is a prototype section (PSECT) in TSS. And it's called "the PLT and GOT" in the SunOS 4.x and ELF mechanisms.

Only the piece containing the code is shared. I think both the code section and the PLT+GOT section are mapped directly from the position-independent executable or shared library file; I think they're mapped with MAP_PRIVATE, but that really means mapped copy-on-write, so if nothing is changed in a page, the page is shared (with the backing store being the file), and if something is changed in a page, the first time it's changed a copy is made, with the copy being private to the address space of the process (with the backing store being "swap space"). In effect, this means that the piece containing the code is shared (unless it's in a shared library built from non-PIC objects, in which case it gets relocated at start-up time, and thus copied-on-wrie, so you shouldn't do that) and the piece containing the PLT+GOT isn't (except for pages that, somehow, don't contain anything needing relocation).

As far as I know, this also applies to macOS (and other Darwin-based OSes); I can't speak for AIX.

So presumably, unless AIX works differently and requires shared library relocation, "or some systems with virtual memory, programs contain address constants that refer to fixed (after loading) virtual addresses" is not referring to any current UN*X. Is it referring to Windows, or to some other OS not mentioned in this section? Guy Harris (talk) 06:08, 19 July 2024 (UTC)

The sentence Every program is in two pieces, one of which contains read-only code and no address constants. is the second sentence of the paragraph beginning In Multics and TSS/360, there is no need for a DLL; everything is dynamic.. As such, it refers specifically to Multics, TSS/360 and the later TSS/370, and definitely not to Unix-like systems, although there might be similarities. It might also be true for, e.g., PRIMOS, but I don't know. References in § Multics and § TSS give more detail.

My concern is that statements describing implementation techniques be placed in context, either by being in a section specific to the relevant systems or by explicitly referencing the systems to which they apply. Of course, it is legitimate to compare and contrast techniques used in different systems, as long as it remains clear what is general and what is specific. -- Shmuel (Seymour J.) Metz Username:Chatul (talk) 14:30, 19 July 2024 (UTC)

My concern is that statements describing implementation techniques be placed in context, either by being in a section specific to the relevant systems or by explicitly referencing the systems to which they apply.

1. The original comment said (However, for some systems with virtual memory, programs contain address constants that refer to fixed (after loading) virtual addresses, and I know of no sources that refer to them as PIC.). That doesn't say anything about implementation descriptions being put in context, and it doesn't appear to refer to any systems described here. Were you concerned that some additional systems that do work that way will be added to the article, even though their code is not position-independent in the sense of position-independent code in the systems described here?
2. The article appears already be structured to put statements describing implementation techniques in sectons specific to the relevant systems or groups of systems. We could emphasize it further by putting all those sections underneath an "Implementation" section. Guy Harris (talk) 20:02, 19 July 2024 (UTC)

The context is the lead, § History and § Position-independent executables. The first sentence in the lead, In computing, position-independent code (PIC) or position-independent executable (PIE) is a body of machine code that, being placed somewhere in the primary memory, executes properly regardless of its absolute address.,is not true in general; it implies that any program in a demand paging operating system is PIC. Maybe In computing, position-independent code (PIC) or position-independent executable (PIE) is a body of machine code that, being placed somewhere in logical or physical memory, executes properly regardless of its logical and physical address.?

Anither issue is that "absolute code" links to the irrelevant page Memory address . -- Shmuel (Seymour J.) Metz Username:Chatul (talk) 09:29, 21 July 2024 (UTC)

Logical memory redirects to logical address, which:

1. has no sources;
2. speaks of several types of address, including network addresses, in the first sentence, but then mostly speaks of primary memory addresses;

and, unfortunately, the term "logical address" is used, in 32-bit x86, to refer to the addresses that are looked up in the page table to either find the physical address or take a page fault (a "virtual address" is a segmented address, and that gets translated to a "linear address"), so using that term could cause some confusion. ("Virtual address" also has problems, as there's at least an implication that not all virtual addresses have physical addresses, so some might think, for example, that an address mapped with base/bounds registers isn't a virtual address.)

Perhaps just speaking of memory addresses, as in

In computing, position-independent code (PIC) or position-independent executable (PIE) is a body of machine code that executes properly regardless of its memory address.

will handle all cases. Guy Harris (talk) 19:08, 21 July 2024 (UTC)

Possibly including a footnote:

In computing, position-independent code (PIC) or position-independent executable (PIE) is a body of machine code that executes properly regardless of its memory address.

Or is the added text better inline? -- Shmuel (Seymour J.) Metz Username:Chatul (talk) 13:49, 22 July 2024 (UTC)

The added text seems more like a rationale for PIC.

And it may or may not be the typical case. On a UN*X system without address space layout randomization (ASLR), most programs may well happen to load the main system library at the same address; other libraries might not be loaded there. At least as I remember from SunOS 4, the address space had the stack at the top, and started loading anything mmap()ped without a specified address (which is how almost all mmap()s are done) at the highest available address. When something is newly mapped, the highest address in the mapped item is just above the lowest address in what's mapped right above it. Any process with the default "stack limit" - maximum stack size - would get the first mapped item mapped at the same address, and the sequence of items mapped by a process linked only with a dynamic version of the system library (libc on most systems) would map the same set of items in the same sequence at startup time, so they'd all end up at the same addresses.

That's not guaranteed, however - not all processes necessarily have the same stack limit, and not all processes are linked with the same set of shared libraries - so position-independent code was necessary in order to allow library code to be shared. PIC also enables ASLR without having to modify code at load time, taking extra CPU time and extra copy-on-writes that un-share pages. ASLR is being done more often, as a security measure, so code, including main program (executable image) code, getting loaded at different addresses in different processes, even different processes running the same code, is becoming more common.

So I'd indicate that as a reason why PIC is done. Guy Harris (talk) 19:44, 22 July 2024 (UTC)

Notes

References