0001801: xargs: add -P option - Austin Group Defect Tracker

Notes
(0006657) Don Cragun (manager) 2024-02-15 16:52 edited on: 2024-02-15 16:59	Change line 123162 from: [-s size] [utility [argument...]] to: [-P maxprocs] [-s size] [utility [argument...]] Add at line 123252: -P maxprocs Parallel mode: execute at most maxprocs invocations of utility concurrently. If the value of maxprocs is non-positive, the behavior is unspecified. Remove the FUTURE DIRECTIONS entry added by 0001811.

(0006660) kre (reporter) 2024-02-16 11:31	Please, I thought the bad old days of just copying man page descriptions were beyond us - things really need to be better than this. There's not nearly enough description of what is supposed to be happening with this option to expect an implementation to do what is expected (whatever that is). For example, how are the args read from stdin supposed to be apportioned amongst the parallel invocations of the utility? One might expect that perhaps the idea is to read enough args until the command line limit is reached, and then start reading more for the next parallel invocation. But that means that if there happen to be not all that many args, then perhaps no parallelism would be achieved, when if less args had been allocated to each invocation, more parallel instances could have been invoked. Or, we could allocate the args to intended utility invocations round-robin, first arg read to the first invocation, 2nd to the second, ... until there have been maxprocs args read (assuming there are that many on stdin) after which we add a second arg to the first invocation, etc. This avoids the issue mentioned for the previous style, but does mean that xargs doesn't start invoking anything for longer - if utility is fast enough, it might be able to process its args faster than xargs is able to read from stdin and prepare the args for the subsequent process - resulting in an overall slowdown from running in parallel, rather than a speedup. Beyond that, "execute ... concurrently" might be read as meaning that all the (up to maxprocs) invvocations should be made to run at the same time. To achieve that, the implementation would need to start them all at the same time, otherwise the first started might finish before the last is ready to commence, so they wouldn't be running concurrently. I doubt that's what is intended, but perhaps. And even more, how are these parallel invocations expected to interact with the CONSEQUENCES OF ERRORS section of the spec (page 3603 in D4 of I8). That is, if an invocation of utility does exit 255, or is killed by a signal, how is xargs supposed to terminate without processing any further input, when it has already processed more, and started more invocations of the utility? And what is to be done with those other invocations still running when one exits in one of those ways - are they to be killed, orphaned, or is xargs to wait for them to finish before terminating? In that latter case, should more diagnostic messages be written if more of the invocations also exit 255, or via a signal? There may be more issues I haven't yet realised. This needs to go back to the drawing board and start all over again, with a much more comprehensive and standards worthy wording added. But there's no hurry, as this is to be an Issue 9 change, there is perhaps a decade or two before an actual resolution is needed.

(0006670) gabravier (reporter) 2024-02-21 00:20	> For example, how are the args read from stdin supposed to be apportioned amongst the parallel invocations of the utility? One might expect that perhaps the idea is to read enough args until the command line limit is reached, and then start reading more for the next parallel invocation. But that means that if there happen to be not all that many args, then perhaps no parallelism would be achieved, when if less args had been allocated to each invocation, more parallel instances could have been invoked. All of GNU findutils, FreeBSD, OpenBSD, Illumos, BusyBox and Toybox read enough arguments until the command line limit is reached, i.e. `seq 1 100 \| xargs -P10 -n70` prints in 2 processes (the first using 70 arguments and the second using 30 arguments) and thus results in two lines of output, rather than being printed in 10 lines as would be expected if split between 10 processes. > Beyond that, "execute ... concurrently" might be read as meaning that all the (up to maxprocs) invvocations should be made to run at the same time. To achieve that, the implementation would need to start them all at the same time, otherwise the first started might finish before the last is ready to commence, so they wouldn't be running concurrently. I doubt that's what is intended, but perhaps. I don't exactly get what you mean by "run at the same time". Are you implying this would require an xargs implementation to use some kind of implementation-specific system call that would execve an array of processes all at the same time ? (I am not aware of any system call that would accomplish this). I don't really understand what this objection is about, exactly, or how any such thing would be observable to any program. None of the implementations am I looking at do anything particularly unique in this regard, from what I can see. > And even more, how are these parallel invocations expected to interact with the CONSEQUENCES OF ERRORS section of the spec (page 3603 in D4 of I8). That is, if an invocation of utility does exit 255, or is killed by a signal, how is xargs supposed to terminate without processing any further input, when it has already processed more, and started more invocations of the utility? For xargs to "undo processing" of further input if it has already processed it would require xargs to implement time-travel. I have not yet been able to find an implementation with this capability, so I would recommend against trying to impose such a requirement, although of course xargs must stop processing any input it has not processed yet. > And what is to be done with those other invocations still running when one exits in one of those ways - are they to be killed, orphaned, or is xargs to wait for them to finish before terminating? In that latter case, should more diagnostic messages be written if more of the invocations also exit 255, or via a signal? Pretty much every implementation differs in this regard: - GNU findutils writes a diagnostic and waits for other invocations to finish before terminating. If another invocation also exits with 255, it writes another diagnostic and then proceeds to immediately invoke undefined behavior by calling exit within an atexit handler. On my machine that results in it exiting after printing that second diagnostic (leaving the remaining invocations orphaned) - FreeBSD writes a diagnostic and waits for other invocations to finish before terminating, and prints more diagnostics if other invocations also exit with 255 - OpenBSD and illumos write a diagnostic and immediately exit, leaving the other invocations orphaned - BusyBox has the same behavior as GNU findutils has on my machine (prints diagnostic, waits, prints another diagnostic if another invocation also exits with 255 and exits then) but manages to do so without invoking undefined behavior, at least - Toybox writes a diagnostic and then proceeds to continue execution as though everything is fine (it does this without -P too so that seems clearly just non-conforming...) I do agree the wording is not enough as-is, though. Personally I would recommend rewriting parts of the description to describe xargs as waiting until less than N invocations are currently running (where N is the value of -P, with a default of 1) before it starts another process, although I have no idea what exactly should be specified for cases where xargs would normally exit without processing any further input, given practically every single implementation behaves differently, except that I would exclude the behaviors from the GNU, Busybox and Toybox implementations, which seem clearly broken to me.

(0006672) gabravier (reporter) 2024-02-21 16:49	I'll add that with regards to the behavior when a process exits with exit status 255 or is killed by a signal, toybox has fixed their non-conforming behavior and now behaves like FreeBSD.

(0006675) kre (reporter) 2024-02-25 06:26	Re Note: 0006670 - thanks for investigating the various implementations, I don't have access to almost any versions of xargs to test against. wrt: I don't exactly get what you mean by "run at the same time". What I meant was, as I understand how most implementations work, there's no guarantee when -P is used, that more than one invocation of utility will ever run at once. Consider searching a whole (huge) source tree for some particular rare operation, piping the found file names to xargs, with some (quick) operation to be performed upon each discovered file. It is quite possible (even likely) that the first invocation of the utility might complete before xargs has collected sufficient args to begin execution of the next one. But as worded, the proposed resolution says to "execute ... concurrently", which might be read as forbidding that behaviour. But no, by "at the same time" I didn't mean to require some new variant of exec() (and fork()) (or posix_spawn) which could begin execution of all of the invocations at the same instant - rather just that it could be read as requiring the implementation to collect sufficient args to (if there are enough of them available) be able to make the arg lists for maxprocs invocations, and once all of those args are collected then run a fork()/exec() loop to start all maxprocs (or less, if there are insufficient args) invocations of the utility, one after another, so they are all running in parallel, rather than consecutively which might happen otherwise. If someone were to implement xargs (with -P) by allocating the first arg to invocvation 1, the 2nd arg to invocation 2, ... until the maxprocs'th arg to invocation maxprocs (and then the next arg to invocation 1 again, and so on) then this concurrent execution would be a natural outcome - and that feature could be interpreted as being a requirement of the specification as written, because of the "execute concurrently" wording. Note that in all of this, I am not suggesting any specific way that xargs should work, the way that current implementations of -P work is almost certainly just fine (and where they differ, the standard can just allow different behaviours, or pick one, or one of a few) - my complaint is entirely with the extraordinarily sloppy proposed (even approved!) wording of the change. Word the thing properly, leave nothing (that anyone can think of anyway) as implicitly unspecified, and it all should be good. I don't think I'd change the wording like you suggested in the final paragraph of the note however, instead I'd say something more along the lines of "When -P is used, when a utility is invoked, xargs shall not wait for that utility to complete before continuing to collect more args and invoke the utility again, provided that no more than maxprocs invocations of the utility are invoked, after than more should not be invoked until one of the earlier ones terminates" (which as written here is horrid, it needs wordsmithing, but it gets the point across). Don't forget to make it clear whether once maxprocs invocations have started, and xargs is waiting for one of them to finish, is it allowed to be collecting more args from stdin so it is ready to start a new invocation of utility as soon as one of the running ones finishes? Or must it just wait, and only start collecting args again after one of the existing invocations completes. It is clear that without -P it isn't allowed to keep collecting args, when it invokes the utility, it is required to just wait for that to exit before doing anything else (otherwise the exit(255) behaviour cannot be implemented). The spec also needs to be clearer what "at most maxprocs" means - read literally any implementation could implement -P by simply ignoring it, running just 1 invocation of utility at a time, since all that is required is not to run more than maxprocs, which is required to be a positive (integer I assume, though the proposed spec doesn't say that - is "xargs -P 3.7 ..." meaningful, or even "xargs -P seven" ?) and so 1 is certainly "at most maxprocs" for any positive integral value. I rather suspect the "at most" is so that the implementation isn't required to run more invocations of the utility than are required for the number of args collected. That is xargs -P 5 echo < /dev/null should still run "echo" once (no -r is given), and isn't expected to run echo 5 times in this usage. All this needs to be clarified.

(0006676) kre (reporter) 2024-02-25 06:38	Oh, one other thing (or two) - if one were to run xargs -P 100 whatever and when attempting to start the 51st invocation the system returns EAGAIN to the fork() call, is that to be treated as an error, or that perhaps CHILD_MAX is 50, and so no more than that number of children can be created (so imposing a silent upper bound on maxprocs) or something different? Also, please make it very clear whether xargs -P 1, and xargs with no -P option given at all, are required to operate identically or whether when -P is given (regardless of the value of maxprocs) the rules (particularly relating to exit(255) etc) change.

(0006685) gabravier (reporter) 2024-02-26 01:11 edited on: 2024-02-26 01:23	> But no, by "at the same time" I didn't mean to require some new variant of exec() (and fork()) (or posix_spawn) which could begin execution of all of the invocations at the same instant - rather just that it could be read as requiring the implementation to collect sufficient args to (if there are enough of them available) be able to make the arg lists for maxprocs invocations, and once all of those args are collected then run a fork()/exec() loop to start all maxprocs (or less, if there are insufficient args) invocations of the utility, one after another, so they are all running in parallel, rather than consecutively which might happen otherwise. I don't see how that would result in the invocations necessarily running in parallel - a process created via `fork` could plausibly go all the way though `exec`-ing a program and have that program execute in its entirety before its parent ever gets to do another `fork`. The only way I could imagine for a program to detect whether an `xargs` implementation does as you suggest it could do would be to check in some way if `xargs` has already read further input for its execution or not. > Don't forget to make it clear whether once maxprocs invocations have started, and xargs is waiting for one of them to finish, is it allowed to be collecting more args from stdin so it is ready to start a new invocation of utility as soon as one of the running ones finishes? Or must it just wait, and only start collecting args again after one of the existing invocations completes. It is clear that without -P it isn't allowed to keep collecting args, when it invokes the utility, it is required to just wait for that to exit before doing anything else (otherwise the exit(255) behaviour cannot be implemented). I am not so sure that "It is clear that without -P it isn't allowed to keep collecting args, when it invokes the utility, it is required to just wait for that to exit before doing anything else (otherwise the exit(255) behaviour cannot be implemented)", given that: (echo a; echo b; echo c; echo d >/dev/tty) \| xargs -n1 -- sh -c 'sleep 1; echo "$0"' systematically results in d being printed on my terminal before any of a, b or c with all xargs implementations I could test (GNU findutils, FreeBSD, OpenBSD, NetBSD, Illumos, Busybox, Toybox, DragonFlyBSD, MidnightBSD and GhostBSD). Is every existing implementation non-compliant in this regard ? > The spec also needs to be clearer what "at most maxprocs" means - read literally any implementation could implement -P by simply ignoring it, running just 1 invocation of utility at a time, since all that is required is not to run more than maxprocs, which is required to be a positive (integer I assume, though the proposed spec doesn't say that - is "xargs -P 3.7 ..." meaningful, or even "xargs -P seven" ?) and so 1 is certainly "at most maxprocs" for any positive integral value. I don't see what would stop an implementation from simply ignoring the -P option given that I cannot think of a standards-conforming way for an application to distinguish between: - xargs ignoring the -P option - xargs being particularly slow that day (perhaps it got unlucky with scheduling during the entire runtime of each invocation of the utility ? or perhaps it was written in Ruby :p) Also, with regards to the argument being an integer, I suppose this isn't really being argued as anything more than a nitpick, but I can confirm almost every implementation I could find (GNU findutils, FreeBSD, OpenBSD, NetBSD, Illumos, Busybox, Toybox, DragonFlyBSD, MidnightBSD and GhostBSD) parses the argument with one of atoi, strtol, strtoul or strtonum which only parse integers, with the one exception I could find, toybox, parsing number integers with a home-made atolx function which wraps strtoll and appears to accept such inputs as e.g. `-P4k` as equivalent to `-P4000` - that function is always used, seemingly universally, to parse numeric arguments to every utility toybox implements. > Oh, one other thing (or two) - if one were to run > > xargs -P 100 whatever > > and when attempting to start the 51st invocation the system returns EAGAIN to the fork() call, is that to be treated as an error, or that perhaps CHILD_MAX is 50, and so no more than that number of children can be created (so imposing a silent upper bound on maxprocs) or something different? Handling of this, alike to the "what happens on exit status 255" issue, also diverges a lot between implementations: - GNU findutils decides EAGAIN means it should wait for 1 invocation to return before retrying (unless no children are currently executing, in which case it treats that as a fatal error in response which it prints a diagnostic and exits) - FreeBSD, DragonFlyBSD, MidnightBSD and GhostBSD consider any fork (well, to be precise, they use vfork, but that has the same behavior on errors) failure to be a fatal error handled similarly to a child returning 255, meaning they print a diagnostic and wait for other invocations to finish before terminating and print diagnostics if other invocations also exit with 255 - OpenBSD, NetBSD, Busybox and Toybox consider any fork (though they also use vfork) failure to be a fatal error, meaning they print a diagnostic and exit immediately, leaving existing invocations orphaned - illumos decides EAGAIN means it should wait 1 second before retrying (and yes, if fork() always returns EAGAIN that results in an infinite loop)

(0006686) kre (reporter) 2024-02-26 07:30	Re Note: 0006685 You're right, there is no way to guarantee that multiple child processes actually run in parallel, or not without the participation of code within those processes anyway - but this is just another argument for fixing the language used, lest someone claim the standard is demanding unimplementable behaviour. Note that I am not attempting to have the standard force implementations to make actual parallel execution possible, the aim here is to fix the wording in the standard so that no-one can read it as potentially demanding anything like that. \|\| I am not so sure that "It is clear that without -P it isn't allowed \|\| to keep collecting args, I might have been mistaken there, but your test is of the wrong thing. A correct test would be more like (echo a; echo b; echo c; echo d) \| { xargs -n1 -- sh -c \ 'sleep 1; echo "$0"; case "$0" in (b) exit 255;; esac'; read v; echo Read "$?" "$v"; } xargs clearly cannot control what happens to processes executing in a pipeline feeding its input, that "echo a; echo b; echo c" all run very quickly, place a\nb\nc\n into the pipe, and then "echo d >/dev/tty" runs, before whatever is on the right side of the '\|' even starts is no surprise at all, what is there, xargs, or something else, makes no difference to this at all. However I was probably misreading what is in the standard as it is now, what it says is: and exit without processing any remaining input. which I was reading as meaning "without reading any more input", in much the same way as the shell's "read" builtin is not permitted to read more than a single record (termininated by the end delimiter, normally a \n) from stdin. But that's not what it says, it just says processing of the input is not permitted (and in the above test, the final "read" reads "EOF" after 'a' and 'b' are output by the xargs sh script - "c" and "d" were apparently read from the pipe by xargs, but never "processed". Since the only visible processing of the args (all that it is possible to observe externally) is when they are passed as args to an invocation of utility, it now seems obvious that there is no rule on what xargs can do (with or without -P) as preparation for the next invocation(s) to start, while waiting for an earlier one to complete. Lastly for here, you're right, the spec of what the arg can be is a nitpick, but so are all of my comments - the point is just to get the language in the standard to be precise as to what will work, and not leave anything to the "well that's obvious, it must be...." as it turns out that while there are sometimes things that are obvious to everyone, it turns out not infrequently that the obvious interpretation means different things - everyone agrees that there is just one possible meaning, but they differ on what that meaning is. We need to avoid that. There's very little in the standard that prevents extensions, so if toybox wants to allow integer args to have a scaling factor appended, that's fine. The point of the standard is that users (applications) cannot rely upon that working everywhere - the standard needs to be very clear on what can be relied upon (so if some implementation found a meaning for running 7.5 parallel invocations, and so implemented non-integral "maxprocs" that would be OK - but not something users can rely upon, if the spec says that "maxprocs" must be an integer > 0 (the -P 0 case is the subject of a different issue - negative values would be some different extension, perhaps). But if it doesn't say what the implementations must support, how is the user supposed to know what will work. Guess?

(0006699) geoffclare (manager) 2024-03-01 11:40 edited on: 2024-03-01 11:42	Suggested new changes ... On page 3600 line 123162 change: [-s size] [utility [argument...]] to: [-P maxprocs] [-s size] [utility [argument...]] On page 3600 line 123166 change: The xargs utility shall then invoke the constructed command line and wait for its completion. to: The xargs utility shall then invoke the constructed command line and, if the -P maxprocs option is either not specified or maxprocs is 1, wait for its completion. After page 3600 line 123175 add a new paragraph: If the -P maxprocs option is specified and maxprocs is greater than 1, xargs shall wait for the completion of utility invocations when necessary to limit the number of its unwaited-for child processes to a maximum of maxprocs. When an end-of-file condition is detected on standard input, xargs shall wait for the completion of any remaining unwaited-for child processes. If repetition of the sequence described above stops because an invocation's exit status was 255, xargs shall wait for the completion of any other invocations that it has executed and then exit as described in CONSEQUENCES OF ERRORS. After page 3601 line 123232 add: -P maxprocs Parallel mode: execute at most maxprocs invocations of utility without waiting for any previous executions to complete, then execute each additional invocation (if any) after a previous execution has terminated. Specifying a maxprocs value of 1 shall be equivalent to not specifying the -P maxprocs option. If maxprocs is greater than 1, the manner in which the arguments read from standard input are distributed between the utility invocations is unspecified, and the number of initial invocations of utility shall be fewer than maxprocs only in the following circumstances: All of the input arguments have been passed to executions of utility without reaching the maxprocs limit. The xargs utility is unable to create sufficient child processes for the same reasons that the fork() function would fail with [EAGAIN]. An implementation-defined limit is reached; this limit shall be not less than the value of {_POSIX_CHILD_MAX}. If the value of maxprocs is non-positive, the behavior is unspecified. After page 3603 line 123324 section xargs (CONSEQUENCES OF ERRORS), add a new paragraph: If xargs is unable to create a child process for the same reasons that the fork() function would fail with [EAGAIN], whether or not this is treated as an error depends on how many unwaited-for child processes it has: If xargs has at least one unwaited-for child process, this condition shall not be treated as an error and xargs shall wait for one of those child processes to complete before another attempt to execute utility is made. If xargs has no unwaited-for child processes, this condition shall be treated as an error. On each attempt to execute utility, xargs may retry the creation of a child process multiple times before deciding that it is unable to do so. Remove the FUTURE DIRECTIONS entry added by 0001811.

(0006702) gabravier (reporter) 2024-03-03 18:44 edited on: 2024-03-03 18:46	w.r.t. https://austingroupbugs.net/view.php?id=1801#c6686 [^] > I might have been mistaken there, but your test is of the wrong thing. > A correct test would be more like > > (echo a; echo b; echo c; echo d) \| { > xargs -n1 -- sh -c \ > 'sleep 1; echo "$0"; case "$0" in (b) exit 255;; esac'; > read v; echo Read "$?" "$v"; > } > > xargs clearly cannot control what happens to processes executing in a pipeline feeding its input, that "echo a; echo b; echo c" all run very quickly, place a\nb\nc\n into the pipe, and then "echo d >/dev/tty" runs, before whatever is on the right side of the '\|' even starts is no surprise at all, what is there, xargs, or something else, makes no difference to this at all. My test was indeed wrong - I got confused about how pipes work, sorry for that. Using your corrected test gives these results: ``` a b xargs: sh: exited with status 255; aborting Read 1 ``` on every single implementation I could test (GNU findutils, FreeBSD, OpenBSD, NetBSD, Illumos, Busybox, Toybox, DragonFlyBSD, MidnightBSD, GhostBSD), except for slight differences as to the wording of the error message between the implementations. This makes sense to me as all implementations whose code I've checked use buffered I/O and do not ensure they only read a single item from the input before processing or anyhting like that, and their buffer is evidently large enough to contain the entire string `"a\nb\bc\nd"` and thus read it in its entirety before starting any invocation, as the tests above evidence (since the script's `read` fails due to immediately getting EOF because the whole pipe was already read by `xargs`) I also agree with your later conclusion that the behavior on all the existing implementations is also likely conforming - as opposed to a reading of a strict mandate they don't read any more input than necessary. > Lastly for here, you're right, the spec of what the arg can be is a nitpick, but so are all of my comments - the point is just to get the language in the standard to be precise as to what will work, and not leave anything to the "well that's obvious, it must be...." as it turns out that while there are sometimes things that are obvious to everyone, it turns out not infrequently that the obvious interpretation means different things - everyone agrees that there is just one possible meaning, but they differ on what that meaning is. We need to avoid that. I agree that it is important to have a clear standard with precise wording that doesn't lead to ambiguities, even for "nitpicky" objections. If you're wondering why I'm going through every xargs implementation I can find whenever you talk about an ambiguity in the standard, it's not to try to "refute" what you're pointing out or anything like it, it's largely simply to try and evaluate how easy figuring out what we want the standard to contain is: if all implementations do the same thing that's probably what the standard should mandate, whereas if implementations wildly diverge in behavior we probably want to think a little bit more about it and the objection might not be anywhere as "nitpicky" as one might have thought it to be. Finally, w.r.t. https://austingroupbugs.net/view.php?id=1801#c6699 [^] , I agree with the proposed changes - they don't seem to mandate much of anything that could realistically change behavior in existing implementations that programs might have been relying on, the requirements seem pretty sensible, and the wording seems pretty clear to me.

Issue History
Date Modified	Username	Field	Change
2024-01-25 21:39	mohd_akram	New Issue
2024-01-25 21:39	mohd_akram	Name	=> Mohamed Akram
2024-01-25 21:39	mohd_akram	Section	=> xargs
2024-01-25 21:39	mohd_akram	Page Number	=> 3600-3601
2024-01-25 21:39	mohd_akram	Line Number	=> 123162, 123252
2024-02-15 16:47	geoffclare	Relationship added	related to 0001811
2024-02-15 16:52	Don Cragun	Note Added: 0006657
2024-02-15 16:53	Don Cragun	Status	New => Resolved
2024-02-15 16:53	Don Cragun	Resolution	Open => Accepted As Marked
2024-02-15 16:55	Don Cragun	Note Edited: 0006657
2024-02-15 16:55	Don Cragun	Tag Attached: issue9
2024-02-15 16:59	Don Cragun	Note Edited: 0006657
2024-02-15 17:01	Don Cragun	Final Accepted Text	=> See Note: 0006657.
2024-02-16 11:31	kre	Note Added: 0006660
2024-02-21 00:20	gabravier	Note Added: 0006670
2024-02-21 16:49	gabravier	Note Added: 0006672
2024-02-25 06:26	kre	Note Added: 0006675
2024-02-25 06:38	kre	Note Added: 0006676
2024-02-26 01:11	gabravier	Note Added: 0006685
2024-02-26 01:23	gabravier	Note Edited: 0006685
2024-02-26 07:30	kre	Note Added: 0006686
2024-02-26 10:17	geoffclare	Status	Resolved => Under Review
2024-02-26 10:17	geoffclare	Resolution	Accepted As Marked => Reopened
2024-03-01 11:40	geoffclare	Note Added: 0006699
2024-03-01 11:42	geoffclare	Note Edited: 0006699
2024-03-03 18:44	gabravier	Note Added: 0006702
2024-03-03 18:45	gabravier	Note Edited: 0006702
2024-03-03 18:46	gabravier	Note Edited: 0006702
2024-03-03 18:46	gabravier	Note Edited: 0006702
2024-03-03 18:46	gabravier	Note Edited: 0006702
2024-03-25 15:26	geoffclare	Final Accepted Text	See Note: 0006657. => See Note: 0006699
2024-03-25 15:26	geoffclare	Status	Under Review => Resolved
2024-03-25 15:26	geoffclare	Resolution	Reopened => Accepted As Marked
2024-04-15 16:04	geoffclare	Relationship added	related to 0001812
2024-07-11 16:32	eblake	Relationship added	related to 0001628
2024-08-15 20:03	salewski	Issue Monitored: salewski

Aardvark Mark IV