Austin Group Defect Tracker

Aardvark Mark IV

Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ Issue History ] [ Print ]
ID Category Severity Type Date Submitted Last Update
0000884 [1003.1(2013)/Issue7+TC1] Shell and Utilities Editorial Enhancement Request 2014-10-16 19:46 2019-06-10 08:54
Reporter rhansen View Status public  
Assigned To
Priority normal Resolution Accepted  
Status Closed  
Name Richard Hansen
Organization BBN
User Reference
Section sh
Page Number 3191-3192
Line Number 106811-106813
Interp Status Approved
Final Accepted Text See Note: 0002531.
Summary 0000884: require sh to set IFS to <space><tab><newline> on startup
Description Scripts almost never explicitly set IFS, so permitting the shell to inherit IFS from the environment is a recipe for hard-to-diagnose bugs (including possible security vulnerabilities).

While I have not done a thorough investigation, all of the implementations I've looked at already set IFS to <space><tab><newline> on startup.
Desired Action On page 2325 lines 73796-73798, change:
Implementations may ignore the value of IFS in the environment, or the absence of IFS from the environment, at the time the shell is invoked, in which case the shell shall set IFS to <space><tab><newline> when it is invoked.

to:
The shell shall set IFS to <space><tab><newline> when it is invoked.


On pages 3191-3192, remove lines 106806-106813 (IFS in sh's Environment Variables section).

On page 3203 lines 107331-107334 (sh Rationale), change:
The KornShell ignores the contents of IFS upon entry to the script. A conforming application cannot rely on importing IFS. One justification for this, beyond security considerations, is to assist possible future shell compilers.

to:
One justification for ignoring the contents of IFS upon entry to the script, beyond security considerations, is to assist possible future shell compilers.
Tags tc2-2008
Attached Files

- Relationships

-  Notes
(0002429)
shware_systems (reporter)
2014-10-30 15:07

 I would think one of the reasons, theoretically, for allowing an import from environment is NLS support, where some locales may commonly use extra separators. Unicode certainly has extra characters it considers white space, ZWNBSP being the most common in use as endian indicator. Resetting IFS in each script created in that environment would be onerous on those developers, I'd think. It's as bad, IMO, as each localized application having to switch from the "C" locale at start up explicitly, rather than use the LC_* and LANG settings.

I would rather see it left as a) 2nd level interactive shells (controlling terminals) optionally inherit, and b) possibly as a new requirement, subshells shall inherit the parent shell's setting. It's also plausible to have an ignore_ifs set -o option.
As a requirement I can see this being plausible for a sh invocation as the system console session, so there is no reliance of IFS being set by the OS, but that's it.
(0002531)
Don Cragun (manager)
2015-01-29 17:49
edited on: 2015-01-29 17:50

 Interpretation response
------------------------
The standard states that implementations may ignore the value of IFS in the environment, or the absence of IFS from the environment, at the time the shell is invoked, in which case the shell shall set IFS to <space><tab><newline> when it is invoked, and conforming applications must be prepared to accept this. However, concerns have been raised about this which are being referred to the sponsor.

Rationale:
-------------
Some old shells did inherit IFS from the environment, but since most shell scripts do not set IFS as one of the steps in their initialization, this creates a security hole. Most, if not all, recent shells initialize IFS when the shell is invoked and do not change IFS in a subshell environment. This is the desired behavior.

Notes to the Editor (not part of this interpretation):
-------------------------------------------------------
Make the changes suggested in the Desired Action.
(0002554)
ajosey (manager)
2015-02-20 12:13

 Interpretation proposed 20 Feb 2015
(0002599)
ajosey (manager)
2015-03-23 12:09

 Interpretation approved 23rd March 2015
(0002678)
mirabilos (reporter)
2015-05-27 21:26

 Please change the Desired Action from…

| The shell shall set IFS to <space><tab><newline> when it is invoked.

… to…

| The shell shall set IFS to <space><tab><newline> when it is invoked, if it is set.

Reason: mksh currently sets IFS to <space><tab><newline> when it was set in the environment at startup (even if it was set to the empty value), but does not set it if unset. An unset IFS is equivalent to <space><tab><newline> per POSIX, so this is functionally identical but permits existing behaviour.
(0002679)
rhansen (manager)
2015-05-27 21:48
edited on: 2015-05-27 21:52

 Allowing the shell to leave IFS unset is not quite functionally identical to always setting it to <space><tab><newline>. Consider a script that does the following:
backup_IFS=$IFS
# do stuff here
IFS=$backup_IFS
In the above case, if IFS was unset before that block of code then it will be set to the empty string after. While I consider that to be a bug in the script, it is a common bug that requires some effort to fix:
newline='
'
backup_IFS=${IFS- 	$newline}
# do stuff here
IFS=$backup_IFS
Because of this, I would prefer to put the burden on implementations to set IFS to <space><tab><newline> than to put the burden on all script writers to ensure that IFS is properly handled.

That being said, I would not be opposed to your proposed change if others feel that compatibility with existing implementations is more desirable.

Because this particular bug report has already been marked as resolved, I would prefer that you submit a new bug report with your suggested change.
(0002681)
rhansen (manager)
2015-05-27 21:57

 Hmm, I just realized that this is tagged for TC2, not Issue 8. Given mksh's current behavior, perhaps it would be better to permit implementations to leave IFS unset for TC2, and file a new bug report for Issue 8 to require implementations to set IFS to <space><tab><newline>.
(0002682)
mirabilos (reporter)
2015-05-27 22:15

 @rhansen, actually, backup_IFS (we call it $saveIFS usually) is a good point. I retract my suggestion, thanks for shining light onto that corner; I will change mksh to match, then.

(We could use $' \t\n' instead of your construct, but I agree it would be an absolutely unnecessary, and probably not often done, burden on script writers. Although I admit I recently find myself to set IFS explicitly instead of saving/restoring it…)
(0002688)
mirabilos (reporter)
2015-05-30 21:01

 Oops. I was just going to change mksh to match, then I saw that it already *does* set IFS unconditionally. I just made a newbie mistake when checking for existence.

So this requirement is just fine and already met. Sorry for the noise…
(0002691)
shware_systems (reporter)
2015-06-02 15:13

 Collateral issues, potentially... Just want to make sure making it a shall instead of may is accounted for in other parts of the standard.

If the environment has IFS unset on shell startup, is the variable IFS the above sets the application's responsibility to export or the shell's, when setting up for an exec or subshell of a utility other than sh? Is this left as implementation-defined whether the shell marks it for export automatically, effectively?

If a parent script exports as readonly a different IFS value, does the attempt to set this default in a subshell fail as violating the readonly constraint, or is IFS considered 'always writable' now and an attempt to mark it readonly should fail?

Does this affect how the make utility is expected to parse the script portions when a target rule is executed?

- Issue History
Date Modified Username Field Change
2014-10-16 19:46 rhansen New Issue
2014-10-16 19:46 rhansen Name => Richard Hansen
2014-10-16 19:46 rhansen Organization => BBN
2014-10-16 19:46 rhansen Section => sh
2014-10-16 19:46 rhansen Page Number => 3191-3192
2014-10-16 19:46 rhansen Line Number => 106811-106813
2014-10-16 19:46 rhansen Interp Status => ---
2014-10-30 15:07 shware_systems Note Added: 0002429
2015-01-29 17:49 Don Cragun Interp Status --- => Pending
2015-01-29 17:49 Don Cragun Note Added: 0002531
2015-01-29 17:49 Don Cragun Status New => Interpretation Required
2015-01-29 17:49 Don Cragun Resolution Open => Accepted
2015-01-29 17:50 Don Cragun Note Edited: 0002531
2015-01-29 17:51 Don Cragun Final Accepted Text => See Note: 0002531.
2015-01-29 17:51 Don Cragun Tag Attached: tc2-2008
2015-02-20 12:13 ajosey Interp Status Pending => Proposed
2015-02-20 12:13 ajosey Note Added: 0002554
2015-03-23 12:09 ajosey Interp Status Proposed => Approved
2015-03-23 12:09 ajosey Note Added: 0002599
2015-05-27 21:26 mirabilos Note Added: 0002678
2015-05-27 21:48 rhansen Note Added: 0002679
2015-05-27 21:50 rhansen Note Edited: 0002679
2015-05-27 21:50 rhansen Note Edited: 0002679
2015-05-27 21:50 rhansen Note Edited: 0002679
2015-05-27 21:51 rhansen Note Edited: 0002679
2015-05-27 21:52 rhansen Note Edited: 0002679
2015-05-27 21:57 rhansen Note Added: 0002681
2015-05-27 22:15 mirabilos Note Added: 0002682
2015-05-30 21:01 mirabilos Note Added: 0002688
2015-06-02 15:13 shware_systems Note Added: 0002691
2019-06-10 08:54 agadmin Status Interpretation Required => Closed

Mantis 1.1.6[^]
Copyright © 2000 - 2008 Mantis Group
Powered by Mantis Bugtracker